{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "训练一个分类器\n",
    "=====================\n",
    "\n",
    "之前我们学习了怎么定义网络，计算loss和更新参数。\n",
    "\n",
    "数据怎么处理？\n",
    "----------------\n",
    "\n",
    "一般地，当我们处理图片、文本、音频或者视频数据的时候，我们可以使用python代码来把它转换成numpy数组。\n",
    "\n",
    "然后再把numpy数组转换成``torch.*Tensor``.\n",
    "\n",
    "-  对于处理图像，常见的lib包括Pillow和OpenCV\n",
    "-  对于音频，常见的lib包括scipy和librosa\n",
    "-  对于文本，可以使用标准的Python库，另外比较流行的lib包括NLTK和SpaCy\n",
    "\n",
    "对于视觉问题，我们创建了一个``torchvision``包，它对于常见数据集比如Imagenet, CIFAR10, MNIST等提供了加载的方法。\n",
    "并且它也提供很多数据变化的工具，包括``torchvision.datasets``和``torch.utils.data.DataLoader``。\n",
    "\n",
    "这会极大的简化我们的工作，避免重复的代码。\n",
    "\n",
    "在这个教程里，我们使用CIFAR10数据集。它包括十个类别：‘airplane’, ‘automobile’, ‘bird’, ‘cat’, ‘deer’,\n",
    "‘dog’, ‘frog’, ‘horse’, ‘ship’, ‘truck’。图像的对象是3x32x32，也就是3通道(RGB)的32x32的图片。\n",
    "\n",
    "下面是一些样例：\n",
    "\n",
    "![cifar10图片示例](cifar10.png \"cifar10图片示例\")\n",
    "\n",
    "\n",
    "训练一个图片分类器\n",
    "----------------------------\n",
    "\n",
    "我们把它分成如下步骤：\n",
    "\n",
    "1. 使用``torchvision``加载和预处理CIFAR10训练和测试数据集。   \n",
    "2. 定义卷积网络\n",
    "3. 定义损失函数\n",
    "4. 用训练数据训练模型\n",
    "5. 用测试数据测试模型\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "1. 加载和预处理CIFAR10\n",
    "^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
    "使用``torchvision``，我们可以轻松的加载CIFAR10数据集。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "import torch\n",
    "import torchvision\n",
    "import torchvision.transforms as transforms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "torchvision读取的datasets是PILImage对象，它的取值范围是[0, 1]。\n",
    "我们把它转换到范围[-1, 1]。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Files already downloaded and verified\n",
      "Files already downloaded and verified\n"
     ]
    }
   ],
   "source": [
    "transform = transforms.Compose(\n",
    "    [transforms.ToTensor(),\n",
    "     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])\n",
    "\n",
    "# 如果下载问题，请先从https://media.githubusercontent.com/media/fancyerii/fancyerii.github.io/master/assets/cifar-10-python.tar.gz\n",
    "# 下载，然后解压。\n",
    "trainset = torchvision.datasets.CIFAR10(root='./', train=True,\n",
    "                                        download=True, transform=transform)\n",
    "trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,\n",
    "                                          shuffle=True, num_workers=2)\n",
    "\n",
    "testset = torchvision.datasets.CIFAR10(root='./', train=False,\n",
    "                                       download=True, transform=transform)\n",
    "testloader = torch.utils.data.DataLoader(testset, batch_size=4,\n",
    "                                         shuffle=False, num_workers=2)\n",
    "\n",
    "classes = ('plane', 'car', 'bird', 'cat',\n",
    "           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们来看几张图片。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "plane  ship  deer   car\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAB6CAYAAACvHqiXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJztfXmQZVd53+/ce9/ee09Pz9Kzz2gZCbQCwmADYpHADtgJIWCXrTjYSqXsip24KsF2VQhO/nAqLjt2le2UChNkmxgMGCMWOwghkMCW0GiXRsuMZkaz9Ta999vfuyd/fN+53/f6ve7pWZiebp9fVde7fe59535nufd9+2estfDw8PDwWP8I1poADw8PD4/LA/9C9/Dw8Ngg8C90Dw8Pjw0C/0L38PDw2CDwL3QPDw+PDQL/Qvfw8PDYIPAvdA8PD48Ngkt6oRtj7jbGvGKMOWqM+cTlIsrDw8PD48JhLjawyBgTAngVwHsBnAbwBICPWWsPXz7yPDw8PDxWi+gSvvtmAEettccAwBjzeQAfArDsCz2fz9u+vr5LuKWHh4fHPz2Mjo6es9YOne+6S3mhbwdwSv1/GsBbVvpCX18f7r333ku4pYeHh8c/PXzqU596fTXX/ciNosaYe40xh4wxh0ql0o/6dh4eHh7/ZHEpL/QzAHao/0e4rQXW2vustbdba2/P5/OXcDsPDw8Pj5VwKS/0JwAcMMbsMcakAXwUwAOXhywPDw8PjwvFRevQrbUNY8yvAvh/AEIAn7HWvnih/Qy955cBAM1Gve2cMUbdb/lz7lBfY23cgeblGyzcsdEXdKCp9VzLFR08hgzCJf3Ll9wYrLqlPl6KMEwlx9MPfbrl3Pa3/Jy6J38G8nvt7hWpn/DQuHNByzUAEPCxCfQ8xy3n9HkDd72+Z+u9W+8ldLiZCfm7oYnV9a39t35DtXSYy2QsQQe+hS/8wZf/qO3Uwd4JAEAcy/e2bB0BAFRrtaQtStHazs5NMbFCd3euBwCQMl1JW5jKAgDOzS0IbVHI360AABp1eQ6iFK+3Gm42R31Uq2UZim3wZU26JpNOzjXrdH0qEmcEE5Z5LBNJW3F+kc4Z+q4Jmsm5hcVJAECpLCrTKKRxBUb25GhzNzQeeOBPkuOuXprvUycrSVu5SGMdGM4lbW9/020AgI+8/24AQEbRETdpnHqJm3U6n8+J9B+EEY+B/n/we19Pzn3z4ccAAOl0IWm7sZ/GvNAnPRcX6PjwK7MAgDAl4/y9//ZbAIAD+0RB8fH/+F8AAMfHRElx9x27AABnx4jGcn0xORfxOjdQTdpGdmToc/cv42JxKUZRWGu/CeCbl9KHh4eHh8flwSW90C8HQv417cjdtnDodtlznWBtO1e2tI/zcdcdYdx1tq1Pd9zCkTKH3plI/lRc8IocerRCX7oPd2/Tfr6lfzdFZsmnuk5z6AGPpYVDd5y546SNzLu7Tl/fiVuOY+JgIsPSjLpG6NDfc/3J3AdO6ukgaCU0qh7sCvtnYPN2AEC1qjjuvs0AgB5FR6FAXOFgdRsA4Ny5seRcV5449K7cQNKWztL1YX42aYt5z2QL1G+TuVAAaNTpuKa49p4e4iwdNw4A6TTNW6NJ0kO1IlJEKuwnOgqDSVu1Ng8AKJeEM+7uqnBfxK1W60Lj7GtnAQCBEdqaDeIsc4VM0gYRGgAAd7ztJ5PjQheN78d+TK5v1nldQlmLgf5eAEA96G07F2Z4/7dIgbRGFbW6TtILeG6373lzcu6DhX0AgKeefiJp60vRfEzMyH56/lWac1Ogtd0xMpKce/o4SSoD22UsP/uv7gEAPPPiq0nblgGSYnp7XgMAjJ57Kjn3+gnq/y1v6E7aDj1DazCyGxcNH/rv4eHhsUHgX+geHh4eGwRrrnJZLc6nYgHaVSrn7UNf36H/1aRF6ERXq8rF9dXBiNvJuLeS6mkFcmxcbW/ToimroPQ9Y9YoiBpEj4VVNLFW5TgDr+IDglaVC5Qx2qlrWlQ0HXRKttFwJ4mutita6RD1jmqLeVwdvixqoxY9zLI4fops++WyqDpOnSVRurdXjIvZHIncUUSdjY2fTc5FERnRshkxvqXTpOIIjRgtG6xiqcWkr9Dr02yQWkUbYkNDfeTz0m+hi9pKJVIdNGOhe2jTVgDA3LzQNjdPRtxQ7Y/x09PcL/UVpER/ErMqJ5WSSSsvknqgUl6eJ3zHu96XHKfSNB9dXUJ3Pk8G2+6cGBzPTU7xPelepZrs62qVxmXV4qV5frdulyDKVIbWxca0X2/aeTA516zTWG65451J28vf+BwA4KlXp5O2gb2kCtmyaz8A4K47RW3zyKHTAIDBU7IuN7/tXQCA23/8rqStXqU5mpt8DgDw4FdOJueCXWRoPnxENuzZmXbHkAuF59A9PDw8NgjWnEPvxG0tPdcZ2hjZfv2FctcXm6SsU38tdFvX1vm+1KCOVyBDGyiXYmpSfv3TKeIAtQEyCJlDV0Y3Z29yHHQzFm6h2XQucNmkLZchI08qEg4zlYr4XsQN6aE1nZFYGfCc0dR9DwACJw3wPQPFZ8i6tEwgAKCquLcUG9czKaEtudytQWtr23UOM9PHAAC1mmb3aXylRXGP62KjaKVSBKDcFwFYljYGN21O2gJ+3DKBzGmlQmOoWDZKKpdDt08adaEjbtA9S2kxaNZ66NjZo61yhZsYnQEAzC+Iy5zbRj2F/qRtYZa400aFDbcp2Sf1BhkBsxkxyjvuvrQoxlOIhyYA4Hd+5w/kH17GjNpPO3duAQDccGB70nby9DjR6G6lJLOIGwf6pI9yTGPv2bInaWvyWs1O05h1hHqZ57uQVfNcIYNn/77rk7bBNHH5O0dIwhke6k3O9RUmmB6hrVYlbr0eCNces+TZ3Udc/jvf8y+Tc4//48MAgBdfeCVp27MVlwzPoXt4eHhsEPgXuoeHh8cGwbpRubSrRLRvs2m/pt1V+TyEtF+/ClvkErqdb3r7ebOSQU7979QfeiyrMQg/d+jR5DgK2V9cq1zcsTJaBmxBFN9xbTClczo6NQhJvI2UyiWb5UhEjqQLQ7lnmnU6kVbDsFplxw6dBsjdk+6fy4nhbHZ2DgDQ09OTtA0NbQIAPPmPjyVt27eT2N7f39dyH6LX0SYqg3pdROOlyAR0fSotY0mFJIKnI5mPfEjz0GR1SdbI49S0dP+0WgPXL2qiznCKk8Cdayjfet7XoTIkV+ukPshl5V4FjkB1EaOxUlPUymzQVKq2wPn7q2hT06Do1VzEfcUyf8Ui3dOoyNlUREbDeAWWcHx6JjnOdFG/g4OydwLeb9959IWk7fQE0SF7XhmJeU0LeVG57N1P0Zhjz76etJ08Reqjep3WJQilDxdpWy6pSFseqn7M0qxy6crRCmV/6YNCBz83zZY5ddQqRwR+FmJ2SNi86+3JuR/Pk4qoHkgU69GXLzjQvg2eQ/fw8PDYIFhzDt1xpIqpENcto7lU607Sh74+CSeUNpP0AX1hy731f6btQEV+LvstRdcyaI/f7HiyralTrppgBU79zPGXk+PubjZeqvwTVTbapFW0aVwnA1GKowMLBeGMm2zQacZCd5FtbU01+Rl2EavVyOUqkxXuafsWciUzTXHH6mMOenBA7nXu3DkAQJ1pjCCcz7FXngcAjGwXw1nQIO7qpWcl2m+gh+gYHzsBAJidEe7Qceaaay8VyWC2ubs9+tblYclmxACaZqmkVpJcJCnmyGsBG6G7JCq0VicaG0UxUIY5uldcFzoc9x122K81NubFat9GAUcnNoXDbFbpvIsUhXKLLDOH3pKnh6WGclloK7OLXRe7T9aUwblWo/7DQNYlzYbusKOTKaGvIHQM7SCLn8kLDznYR/N17KQYk7vZbTIZcqy44BTtmYwqkjNdofOTU8WkLZujsbr9HwQyf728T7oUHeUFmmcdXF7jZ2OR5+/Vo8eSc6nuYe5f9rpzy6yptbW87YOI10dFUecGSMp8z/v/RdK2eeQ6AMDU5MU7aHgO3cPDw2ODwL/QPTw8PDYI1lzlkqRfbflpsUs+tUqkXRyJESy9HIHpkD53qXho25Uuq7A/ttG2FJ0MpZ0QdLC6dooedf2FK/z8xhVRBWR6yWClfc5rFRIrCyrNaIYNns5409slImRxnoyR2nhYYF/vmTkRb8tV6sMGdC6lVDpRivq95vprkraD15Ov74hKdjQ9S6qZmWnyaa6rCM2dO0nVkgqk34nTVPlwc7f4Bu9if+GXX6OIzokx8csHz0OlJL7YqVSG+9iJpaiw+iGjVFYBq0ZqFemjXmP/fVZPpZVvs4sebTREddE0JI7XrKhLQvYxd2qVvn5R22Q46ZZTl9ExXTc/N5e0mYATSfGzUWvIXqg3Y6ZHGWxj6i8wKtq0m5J3laq0tjo6tWlpHpo1USe4e8TB8q+QbCT9T52kyEi9n268k9Y2UhGoSaK2kPZioPpo8HMwq6JToyrNbwyVkIzTGluOmNUOBrPzZHQtqIjVkOetWJV9V+E9WOim62anJYp0kA2l5yYnhe69rIZR+79cdzpKptdoQz2NL47kGe3qJTXMlOr3QuE5dA8PD48NgvNy6MaYzwD4KQAT1tobuW0AwBcA7AZwAsBHrLUzy/WxEpxrls6OKsGB2m2RIxHd93QfCVerDBIdOO2VimR0wsqnV8vKt0c6tvVrWlj0DnR0Ms624s63vik5jjja8Mix15K2bESG0m3DYlxMs4GtwpxDsSac9yJzhT05iUjc3EtcXF+fcENdPWSgCpnj7VUc5rXXUYTczl0SAmfZyHri9RNJW425trjJ3OeiFIDo5XSqkUpD3LOFUppes29/0lZnrrOfOapr9u1Nzi0sEDfbqIkxrbe7h+/Znj9jnDmksjKAFjJsrGsogyZLDTXmBE1G0qmG7N6Yz0pbnTdtVRkjMzy/dU6V22zKBshkOa+Kch11c1VWxtZsxbllWj4n8+ecA6KsPDENZ/DWeXeSPD20y7KhcLCLLGVMzc0nbd081myX0LYU/WkZyxmWjrLKGPmRf/Y2AMAvfew9SVulxPtugKJInzsmY/mLL34JALAtJf1OjHL+GsX9NjndcMAcslHce71Bx6WG7PVa2bl7KhdJNvLPT9EeODclUtVttxE3/p2Hn03axs6cAAC8/73vSNpMxr1eaU7rNen/W9/6ewDA0WMiSX73EXIA+Pgv/HNcLFbDoX8WwN1L2j4B4CFr7QEAD/H/Hh4eHh5riPNy6NbaR4wxu5c0fwjAO/n4fgDfBfCfL4oCx322lD/je6vLnJ7cZQ3U3GoA1pWpxngVuWEudy6X1aPdRuDQKaAiGe0KLpLDQ8IZWw7ouUHprlMZ4qTqEI67FBOXFbNkUymKXrYnSxx9TulII3bTGuwRbnnPbgrscLrDUOW3SDG9Z89I4YfFReLU5pQOuMp6UJePRZs65mZIrx43pXHLZnaHlMsS3XaG3ce2bxNJZHaRcpbUdObIGmcLnBvHUjT4VtoFLc2uZ4GSFJyrZsgSUaWqA03o+nxB2SXKxOWVla42y7piw9x+WevLE65d5dhx9zeyjvPzHCAWcok2dX2K1yOu6XwwPC4dBMY5ViLWWeuKkK6YhYXKf8KBVo3m8nvyfdfIXP3tk/yM1oXu+Ai5Am6963Zpa5KN58QY3bNspADEG978PqZHuOUd+2m+aoreKudridlNtdlQ9gB2x6yXxFVytkRt1bL0u8jBVAsLJAGcE+EEL7xI2Rb7MrKf/uEJKl4xMjKctN1+2xsBAIa1C88elsChhx5/EgBQnBFJq2f/NlwqLlaHPmytHeXjMQDDK13s4eHh4fGjxyUbRS2xtsv+TBtj7jXGHDLGHNJZzzw8PDw8Li8u1m1x3Biz1Vo7aozZCmBiuQuttfcBuA8Atm3b1vbijxM1gs4xknxbtbnk9iz2Kf1KclWHFLValdKx+vuS65fQft7rVquqac3Nsvx1iSdja8HTJZ/tmKi2JIShz5SIqyWnRgjESFeJSPxtsjHNpORcvocMoN0qd0mfS02rXM+OHTsOAKgWSSatV8TYVGGVRVUZm9w8xHG7W2l3N9HbV5A8rBl2BezuFiOdNbQX6kovEFqiybI6o6ZUAd09pHI5tyh0T00RvdKrIM0GUBjlgsZifEOpRFwEqmHjYrUhKpdeHsuCyhkyzcZZrTmbqc+39qWLkrjxauLc+ilD6fQMzfkuzr9aK6rIUr6ZzjHiDNiIZL17+knIfv0EFcKYmJSCGFu3UQrgvkFRCTQrHFEaL8+kfe6H4icxy9GYg0bmr3GO1GTT06J2+OEponN8nlRAs/Oi6zApTvdrJK9Pmg2kabXHunnMLjBYR9rGnO8mUkVA+tkNcl4Zq03k6rRSJ+XJozKwOhkyp0uy16fHKdr5/97/paStr0AG/RS77x55+lBy7s3Xk0H/a9+SaOfGrNLrXCQulkN/AMA9fHwPgK9eMiUeHh4eHpeE1bgt/hXIALrJGHMawCcB/C6AvzbGfBzA6wA+ctEUcACQDixyBQ8C7VblOLDAVYYX40qVgxGaKu9DuKQa/UWRtorvntew2sHlUApydOrRXd9JOlieQ5+EBNkk06aCcQIOtmiq33AXFGJdMXXlgeYO83kJREq5IBm1Lr29xC012EWrWhbD2SK7/alaCQlnrqUlx51m2RUunRZCcmxwDK0KamHjbUpVhE+x+6EzrNbV9Sln/FNSwdQ09VEQT8YEGQ6+0i6KJea0IyUZNtg9rsycWqkqbo5pzo9jFR3FpLiC7N1MjqSRcpn6qBSFo0+uUa6Pac4HM1+Ue80u0LOxjw3fKSWZxXU1+a6Nuc66MvC+eIQ48tdfp6yFIztFSoq6ObugCixqxHR/k5K2pdi8TTj66SM035WMrHvlWnJNfORlGfPpGZq34X76bKSUkZjdagPFjdcbLvOhrG2V180ZbJvquXR27rkFlQuH+9MuprUKrXOpzhKAChgaKHDW0fxg0pbjOh/7d0vRkJ27yTBved9VZ5UjAG8jlTQTNlzeBXS1WI2Xy8eWOfXuS767h4eHh8dlg48U9fDw8NggWPNcLk7v0FKGk8WnWOXByEUk4tVZjCqpqKuQq4anVVRjlcV9q8QzJ+aLmmTlnCsrpb51EXW2Jd9MJ9/3tqa2/jopUnQ6YbOKPDPVSETk5LvKzzhgtYZV6pKlfu36X1f0YmpCclicOkORp0ati/M7d8n+06pWaJ1F9LhFKqebFHqEXpfm1KW31SqXLi5IECtja8hG2Ywen1O7OTWI8lWusticC2V/5AJHVPukltgfOVSL4KJYqyo/jvOpj5mOmtprC2w0zGRlLA1WDwRKv+hUM3Nc87NYUnVS0zSXWZXht8Jqh7ryrZ6do2jKV49SXyPbNiXn6uxvH0OrXuj+xbLshdHJ1rFUm7p/rqEZiwnZFS3RxSOWIhXIGvSEznAsc/r9p8gX/AN33ZG03XItrffWbWTg1fmXZqZcnhnpt1whOhcrssnmFmnMs/N0/eiMjOX0BOlGhvpELWW4kImLUgWAc5N0j64CTX5/TlSa5Unynz+lCni4Zasqh/gXnn4cAFArE219vbL/xoq0HoFVvv0rOG2sFp5D9/Dw8NggWHMOPUxcw3RmO/qVK80Kd7ilnxPOV+gX9qnDkpGsyL/OO3dKfNMgZ9/LqCIFcZOj5vhHv0UqcAeKe3IZG0PFYjaZo3PJ6nVSfMfhGh09xxbHANrAy65kbJTUnL0rGKBNYzFTl1rBbbGpOCrpTohzRSmCUJbclSJzVtpAu3gyHX1dwklv28lck+LQixVeK+aUGtpFkV3UQiUU1JmDrlWU+1+yB9iIpeebXRNDxRkbx0FHysGVx7zAWSeLVenDMMeWKoi723UjnNluXiIGHarch1Hl4ELmrmfnpcp9jQ2evZsoclVz3mW3h2tivIyYG4+Ve+NikfoLuIBGKivzHfLENSFcX8QZGHsGhLPbkSK3wiOvUNTrQL+U94tj4lKLdXGJ6+klY+TijOy7uUWiyXJOlEZdRZvyHm4YGUvM2RvryugqJkXCE4ck18lAP3HERhXOuG4PZbp893uFQ69ytK2LJH71VSkt992Hv8NHKi8SS10NiPuke757e8niPdy/WcY5TdJGQb35pqdoDWol2TP5Bq3lpgxz8rHs19dPc+ZIxVGfOkpG5eqE9NFbpeuaTZq32qyIWodOUfT0L/7yLyZtd72bctt85YGHcLHwHLqHh4fHBoF/oXt4eHhsEKy5ysWJqU2l/wjZsJZVfqFxmdQvg2xAO7hDHIiffZXEl0Pf+570y+L4zr2SRnXn/n0AgL5B9h8NRGxtWLpnrCq3OwOiUf7OxhnKXESnrv3pPlUNw6aL1FNqBKdSspwUyyh/YJdMKa0KdMTJHC2/XA2rEkOx2iNUkY7OEKzrfpgkCxqPU6cf5sr0BRWhuYuLTQTKelpnXYeLCp1fFGVRjaM2Y2UoKnP6h8V58cmt1Fwle1rv4UHx741Y9XP8xHGhjYt1bNkkCcmcz/25Wbr/uSlR16XZrzyTk1Ss/X2s2ujA0rhUs0Fd+zuT2FzolWRbWwbIUBbzPCstBWbKNL5FFU3YlyMf5bRS9YU5GnMmdoZKZbzkfWcDrWrj/aTSxfZwiuHFMiWNOjsuY7/+IBUSOf78aNJ2epRUMyobLmYW2Vi4lZ6JXI9KBcxpcBsqWZl7DGK7vB96KpLnK5umedNpbtMR7YvXj0k93Aqrtly8wnxRVKsnzhwGAAwMiAplcYHmq1yVJGu5NK13vURqta5BmdPGPPUXF0TFe80I7XGjlEbjnDZ3bJoiQMfGlWouT+NamBDaags0v5Ml2QRfe5QMzQ3Wy/bEorKqsurz8R/8IGl74gePAABuuPnNuFh4Dt3Dw8Njg2DtOXQXxKe44Ab/OqcKYtCcOknsRDxLXMjQjuuSc297680AgPG9e5K2Q4/Sr92DDzyQtOW7yYCy/1ri2q+98ebk3JYdBwAA6S7h/B33bbWRk93Fgg4Rj46riOvCkXYxd5XTFdMDui5tiEuIVO4Sy65qgRXj0TwbAReURLEUNWWQc1xt3BJpy59qLIbptWx0CwK5ftcWMjD3KEOpcxcrFqUM29wic6AcZrpQFONUOs0FGhRtDZYCUsqdr7efOKQsS2aBitBscgSoVcbcBabbqLJ7NS4Z9urLRwCIJAAAm4dpLLfeJtLayE6KYjz8msrRkRDOrpUFcTNrcqrUulXGZw6BdZKcHmeNpY6yih41TZIQrO53gdY57SKglYtiyFyt5saTaFR1Lxty2bYs0f3si88n5waHaT+PjgndM7OcXlZJcFGBi4xwSbSmrnfInHZVFfwwLFE0VbrkpYiUZHF2jDjdoa2S1rh/57VE77NHkrYzJ1/n68hw29Un7oI/8XYqhFFR6z5xjjjzWkPlLeK1d1J/tSwSS3WBuOrSouyn3SOUZrpel37DKrkk9mdpz784JRLO1DgdFxdkjxmXOEaVLWzy/HbniI5yRebbGX97e+T6KLXUrHzh8By6h4eHxwaBf6F7eHh4bBCsucolToyFIu47Y6iyiWKafYLPnaRoxS3bpE5lyCk0gx1SSf59H/4wAGD/DdcnbT94kOr4/f1ffgEA8GiPqGP23XgQAPDG28QndsduEtF7u1U6V5dsi9UqrtoOAFT4OKMS+eSdkbMuxsJ6mURvl2417Bb/6BJH3m0fEtVPijU4uV5RGSyFNk7VEstnu8olsMoPnQ2aEatBbCxj6WbL3U37RI3lkmE1OyScWuRQuakZlYCIfdRrao7meT6MToRUJdG1WiaRN6yJeirFxu3uHhGpc3n67iuvvJS0jZ4iVZzzqS8oNdbO3aReeftPvCVpmzzX7n+enJshcXshECPqQIHvn5G1XahyQi2OMDRaX8LRzplQjKjucWuoaEk3bxlOUxyppHMNTjZnlCW7wXOv1UGTU6RG6B8iFWWlLHRPTNBYrrv2pqTt8EuU/lWvY5q3m+ExV5WqqMlpdgNllHcJrRoqKdZSdPXJHh5gFc4brpc9fOdPUB3c7r6hpK3O8QYuGdr8jPj9T47Rmk2pWICRneTLPjsr6znHaj9Xd7VhZQ22X0PqvWJRjNUTizRvs4ui/jjKUak5Z5hWif+qbOxvNGVfN1hNmAnlWY7SdK8MRzsHoVy/UCQ10Gf/8ptJm1PdffKTN+Ji4Tl0Dw8Pjw2CNefQZzga9MhL4rrUwylT926VX+6hQXJB2vEmqj/YNyC//vPMwZwaP520LfIv6vCwuMDd/a53AgB25Ym7+e53v5Gce+ybXwQAPP8P30na3ngzGU1/9d/926Tt5htuAAC8fpzyOTz+D5K0fn6BuJuayrPxAhc1KCoOwlVld9xtWkUHFjiFbD4lBtCwi8Z+8/s+jOXQUEUnmq7+ZSScYOCqv+tIWDYudvF837x3d3Ju9hS5CT43ey5pc0bWeWVwtEtc9qqKu65xMYi6KgoRc+Tn0ICkGa0wN1ZmY2ucF1fJ7i7inqaUG+KRI7RXqlWRegY20X7o4hqe11xzbXJuZIQMcd/4xleStqlpMrJv2rYTSzE5Q+e608IFZ1NMk3IPLXOUZIqNl5GSfioVmpAshDtMhbTOVkVXGo78jF19V8UFV+rEJabENyAJu51dkDwii2w0Hxwmd77urOz5LqYtVsbwXTs4slVJkqUmcfnFGu3XUIVAN6t03KjJfqo5g3qopJIlqMeKcDaaHzsp6/jog/TMbdsmEd5BSN8p5InT7ekVjndmigzYLz5zOGlz7rK9yni6ZQutd3oLjb2k8rxsrZOr62JZHBcmp2jMNSXZdGdJQhkfJWmmUtaFPLhIS1P6aDZpDUpFWT+3P6ZnOKpcu3jW6F5G5di5hEzfCTyH7uHh4bFBsJoCFzsA/DmoELQFcJ+19g+NMQMAvgBgN4ATAD5irZ1Zrp/l0M366d6CcGWvv0jVsV945OGkbccA/SpeN0JcU3pc9GgnR18AAJQqos+rcwX0UOXNaLLeLJOhX9abbhL9epq91ybOCQdx4jDlovjHbwsnP3qMaDvDOtupcam+d/IM/ZofP30maStyWSvNQRtOZB9HzI0oHfreHaTv3bppl4z9GtJ/FkakDS88Cg0M7xM4AAAe1ElEQVStQ3dun4FiCBy/r/O1GM7/0tNN3M1N+0W/OXCQ3DirixJ94vLYLKgiDLMLXNyBpY2iKuPlOO+FWVmrmWmSVM6ekTmqsw7dBV81y6K/PXrkFQDA9IxIOBFzlv39wuUP9NM+uvWWNwAADl5/MDk3PkFrVFYc2MK849DRBud1mlV6+5BLv80r3XLMLqt55uQbZR2YRefSkXDo2YD2eKxy8qR4LCHvhaayY6S4eIRVOVRcdsGiKnBRYy41ZDfKfbsPJOeiBtFdbQpt27eRNGNCsWOMTREdo1MUpJeP5Hlsljl/Uazc7jhYLA6W16FrHb2LRJpTbq2fuZ+eq61DMkd3vOUWAMA8u8MePXFS+mO33dsP3pq0ZUMOThqXwKJPf+3LAIDNQySJWJX7JWKdeFXl6SnXWOpR+9Ty+WKZXpEVta/LJeauVUGKPJfwi9IylpDz3QQJ6y33bFRpLDW1n/Q74mKxGg69AeA3rLUHAdwB4FeMMQcBfALAQ9baAwAe4v89PDw8PNYI532hW2tHrbVP8fECgJcAbAfwIQD382X3A/jpHxWRHh4eHh7nxwUZRY0xuwHcAuBxAMPWWhc+NQZSyVwwsux+9cabxa1q33Yyajz28LeTtmeepsjPJ58m401KibKTYyQCBSqFbDrtVC0q0pGNR849aGRE3BzzXBnezIhYXuKItK98TVQuERtUQxajejIidhXydPwmFvsBwGkn4oyoVfJDdN+QC1p29YpoPzhERpvh7aL+6NvM0XV5bR1rRayMdUk6XCVWZlj67VY1HQeHiKYBFg2fe+Kp5JzLJTMzLUZR56JZaqmOTmNusAvk3LxEkZa5AERDiZVVNi4ZldvGqVzmZkljVy2JSqfALor5gqx3hud8eFiM5m+6jfbPXk7JapTxd2CAjITveMe7kraTp0hEH1fr7dDDhrkeI2oHN5c6T0/JFVpgVYRVRkOXvTejov9yvAdCVWgjYLHdRTWGgbjTpTmaNm7KnNoa0ZRW48tzbpFMls51FSTHTZajPHWBi2qN3SxDMby7bMb1OkenNtU5fg7KKkq2zmq9KFi+DqYztgPifhrqGrw8l++8+4NJ23vffScA4BQb5V++7/7k3Bles7Oj30/aXAGRRWUkfvkoqUNjdq286WaJCH/m2VcBAAtzol6pcG6gKC1rVedU29WY+q8qw36J895kC6Lyy7KbbKTUMGG4pK6xGnvElu58ly6Qs3x67NVi1UZRY0wXgC8D+HVr7bw+Z6k6ckdqjDH3GmMOGWMOlUqlTpd4eHh4eFwGrIpDN8akQC/zz1lr/4abx40xW621o8aYrQAmOn3XWnsfgPsAYNu2bW0vfWdS0QaUTD9xrm+96+6k7eCtbwQATI6SMW30pBhLuthAOTEmbovzMyQ8VFRQS5SmX9HuHuaWeyRrW6GLKNnTK4KGqwIeBGIMyvAvcaVEXNPpo+JCdfANZIi7dv/upO3sFF13/Y/dJffaRXloGpyfRBdqCPk3NlaFIlxF89Au/wveVAUgnNEtq7iFvcylXrtHpJK+QeISquy2deqIZDR0ldIXFDf+/JPkojk6Opa09W8il8oCG1YrVXHlqrFE1KwIxz3PhQsW5qbVdcSVGt4NKeUK183ZHvf1SoDTJi4osVdl0ty6lcZVq7kq8EKHy7FTLKrCHCqwZCl6cxwQolwIXSm3iiqksFAivqbBbqKRKicG47hrWbMaB1p1d4mLneHCLpYlImOES2xytkdd1iybJu47lRcu3xmJXSm/BSUlNTmoRdcXbHJGw0xeOMbXXiM33AZLA9t6RTpxRUm0dJLh4izpFSrVH9yrOFjm1rOqvOCmQZqHE69I7plHmM4dO2ltP/ihn5Fz3yOX4hdfloCy6THO26IkvusOkJTW10d7/nvffzI5d/oMSfgNlflw/z6616Y+GfPUHO2V2jxdFwSyF1IZegcY1dZgqQeRypLKkpuTFkPt4snj1Cbly+C1eH4O3ZC88GcAXrLW/r469QCAe/j4HgBfvQz0eHh4eHhcJFbDob8NwM8DeN4Y8wy3/RaA3wXw18aYjwN4HcBHfjQkenh4eHisBud9oVtrv4/lpYF3XyoBThzRFa+d4aRLRRMWNpF4tv16UlfcovxqndFNG9MWF7mQwoKInxEb/7q4TmakUl06C4BVao2QVSJGGzNcHVBWcTz9mPiDP/4o1QJ85rUnkrY33Eo+sz1bJSKxxmKq5bwmDe0rz0KTTsXqCmyYFVKV1q2I5a4ww0CPGMcOHqDIye6M9DE9SnUQixPk411ZEAPhFB8v6AhUjr5VGiJMTZKh6szpUwCAkvIhr7Bao1bRRS/ovFXr5wpmOFVLWommbom2qIjfvRzR6tYYAI6foPv39NDa6jmdnib1ztGjkip3gf3nt+8WVY6Dq/GqVR0NTn9cUzk9Ao6mrFecSkIepzznP0kplUSd62m63DUAkMmS2st0SMFrYxeBqp4Nnpso1L7VTAfPrU4vO8Dqy2xWVDSzbHw+N3k2aRvsI3obKHAfQqMbVa+qzxulWtUJnZDNKp9svi6l1JclXr/7P/vFpG1y+tMAgHw3fffAflmfIVbvbVEFUAKOjtWVWxa5huzX/45UNJNTYjAdGKR3ygc+LLU8t/TQHn/k248IbfwMjY45pwBZ20yenBjCSBUBidx8qII3ZolRFNoA2j5vPlLUw8PDwyPBmudycdy4LkHnfsd0BXm2dSUctP6lD5nLyebE/S87SJzRQKzMDu4XUxoUJXQca8OjKwqhfzqTrIXU7+3v+cnk1Mh1lCWtvCgZBzdvZSOrKpxh2NgWucyN6mfVwv3Sqzamya7Aodcawk1meMglFZV36AWKph0/JYbPuQniasHX1RSHPsMuh3MlkXDqHCGnOW7nztVgjltLOIkdTrXFSbk7Qcz/1ThjXX/vpuTcbbdT5ODAJjEkznL+n5Kq0u4iKLM5Mr5pA9Q8R4XOLooh1K7Ayzjbnza01djtzxkUAaCL91udy465og+AGCj7+3TOGqKxVJT9EXG+nSDDkp8qR+hKolWqKvskF0sIlZHT5ccpLrCRVhmEM5ksf0q+oLOjZMiPm4qT7yJ6TZbGFNfEgFxIs0Ezai/P2OFxSdCVk+sTd0W1h119jb0HJFy3+DLdd4GNkYdfkeIXuZO0X+dmZU8Wy84FU+4bcUSpW73bOf8TAFx3420AgNqMFKz46oM/BACcOyec/DznW2ryvs7l5fkN8zSXuriNe0fofefOdzznCG590HGp8By6h4eHxwaBf6F7eHh4bBCsucolcKoOLXq4wColmKc5EtGy52bLLxGrVbQvu1UV7JfCqQVaDTpOHNJiD99TNSXVzrnOY6AiL4cPkD+r0SoGltAb6GAsYRr1OJNvatEUS40rncakaOTPSVUI4BwXbSguStvZ0ycAAFPHyafflNsTBWnxPUn/qdUqK9BkWGWmtAMdDbtuHTKcOrhXqSmavO4Tk+K3HnB0oo3F4LjIhsBokdYxm1H+3Kw6SWfEMOhUEZ2QdvVL1djq3EdPj6h+nOE1sYU2ZC/kOGFXX5dECM/HJMZXSpJozDbpOmc8taoObMi6HxeBDADptDOQC70LLm1zmVQRQbpd3XT6lET8jp4lY+iWYaWi5FtUYlr3SKVe7uZUtqFax2rN+WerOqpoRaD0IDlWhTWVCrTEcQq7lcF75yaiqcnFI5o6HoPXoFKVZ/vYGVJfvXxCwmDmeMwf/hnKRnLttVJ/+O8eehAAcErVkp1hFU61rr3CbctnTdW5zdml1yDZKy3PoYvYdmrlWL9bnD4X7W2XAM+he3h4eGwQrD2H3ubaI2jnlZVbl2L7XHpMV1INEE7HdOCM3S9n6x07/Tp2amPO3H1ZV7R3xkur7+no11wt/3I7CUBxMgmNipNxbXYlDl0xFy6lSEO5tjm3qswmMe7sTnOKWc5bcfIFiXo1nA43pXLwJsUCNYcuRBIdLel5OYJSExonE5c05buISx3ZQa6dWVXg4oXDlHtjaLNwccObydBcVxyVM0JmOTJSGxKtk+DUHAWR5sY6o4WfYnLTivN36YHTnPo2UrlRkEgnMk4XLQnl+hgx2xsGLkJY5jvi9LmhmkC5TpcXZAMlt2VUClfHBR898nrSNjpKrqZbh5XE4oQAdjYI1Z607llTBTEMR0Fqjnspzs2Kkb2rxjmQlITmyrbNq4VxZyPOd9NUOYpcpGouLytz4wFaj+EhkZyOsjTyEpcofO5ZKUJT4HH2dysJjcdaUcVZ4NxDeV/lsgV1vZMQlYtzih0dIpGmcllab/fYhup5dBsqUM90sILTw2rhOXQPDw+PDQL/Qvfw8PDYIFh7lQt/nk/lkkiuTrRXiYISFYoSbwPrRJ92Y6sTh1sVKqsTd0InSju/eKN9S7l2oBKtrHUV4XVkpDOKstjaYhSN3UVttJlw+d9fLfnaDkNpsOgaqAjNNKsIhvdfx/9LJODp5ymhUWNBjGmufqRROoBgiYrIdBAhW6LnnHpMjSXFfs6LHFm6WBPzmtPgjE9Kgs9z03Td7Jz4DbtIxIhF34EB8WV3Cb4aqpZnOktj2bxJomkdarwGoZLKKwukusirtLJBknSJx6ZUKXWuBwqVcjaVYhrTMn9uLiOXzEute8TqlaZaM3DUaCy2auRcdSH2UTdW9mSF8zc3rPhuc845RKFsGmfcTDEdWm3TYN/7iooadmOvaR95tCKXE5WOZbqryqAZsnqirlQXdY7ATsP523fwZdf7m5+roT5RhUWcMrjKlZwKOUm419+dZnqkkyarASsliVNIInh5XrTaq8mplLVfuVP1aaPoIsd3NFn1mFKR6U4F29Mlz1zoVS4eHh4eHg5rzqE7PrmldrjRZ/hYwg75QxkkOrj1BR1+q9w3HAPdYs5pT7eQHLb8bi6RJIzRBsK4je7kHjqUzUkZjm51vRM8OuXICGxbk9xHl290c6PrPTpDaU1xQxxIath1bmjkGrmcDX4nDj/edpNOfERH98WwgzE3cG0yvjnmjCps3Lv51jcl57ZtJ0Pp2Ohk0lbiIhnZbnG7c3lgss5dUFWBd66GOh9/04qL5lJkmIurhsJ9ZppEb1YVNGnWiStsskSRSQn37qKAKxWJCs0VyP1PR/zWmft2Eo7tsHlCFRVdZ9bcKu7aMJ0u8tNASRFZulfvZrnn1hHKf9K0EvFbqnA6XDeGUBvq6V4LZeFgXU6juKoM02hFy77mvRAqo2HYQeJM21b3TW38jZmrDnWEJtd6jdTDMdhPbWU27OucP875tV8VlcmzK2ik1i+RJHnv1rURmtMl67TX7pmrKemyxs/L1AztgWpFGV3ZxWBoUPbpNq6BmsHFw3PoHh4eHhsEa86hOz1rrH7OO+rTE67duQbqs47TVRy6aT1H30nYX9ci59ov78yJOr2wExRaqWz7nu0QcGBX0NcneviWL7RLJUvRqGs9a2tAA6Bc9yoqS2WZ71WPWq4BgPzgFvrsFXfBxTkKiOm0Ph05dOdipzgxp6ttCSRzwRusDx2dlAruW7bvAQAMbNqatO3m4guptMzHOLviBSzrRSrvSIN1mH39ole3Rimhl5KdonNK1Y00u9HVVSbDiPkhx9V25cS1rcn5XeZVcFfA10WKs6tw5kUX6KQFM5fNUQf51AxdH6qsmfWYOO2yIZtCNid5W+ZrtGZzNbGFlNiWsKlb7AeWswtaDpZqGsnlUuUsjmWVgTGTInpTKxS4WCyLBOCCgnJKNx/yuLqVm6obvmU7QJBW2SeXBPvQYeITmDS57JMp7i2rcuy4R2h+UdYx79wLNfEsIeQ5Y6RRunznblkpyRylnUuqyozZyzaEru30OTcvEk6RJeBY6fLLnJdGVu/C4Tl0Dw8Pjw0C/0L38PDw2CA4r8rFGJMF8AhIVx8B+JK19pPGmD0APg9gEMCTAH7eWrs0ncN54VQu542SSjyWnEGx0yUdVAGd/umQRkHyu6xMh2mzntq2c50iXFv6cOqjDiqU9qT4IiauRJvOueJsobqeZYPT3DZVNByn7YArR6oD2ZxYaZURK4kUVHQntDlDlR6SUy3oiFVWO+jiB07T49Qx03NiSHzxVYpe7VUpdW2a1C9hXfiRBVYlbeYCBvmMLixBAw0jMXrp3BxLEbO6QbuCOnfPSBkcE1UOG2dbjNasrylVVLX4IrleZrLSh8tV02AjbaTWOGb9S131W2T1R62o15HXltUO5abcc3yWVC7HT0mukzTPc3afRA27iGDj3E/VPnGFTUo1UblkWZ1g4+XVgHp7B6yyaKj0w85dsGiUa2fkIi5Z5aJ0UK4QSqd3hSajxoVJnLNEV07MjO6Z0zV4UylXNET2jFvbWTaoFgrSRzrLr03t5xA4GnUubOfiTGPuLsieT/GzoY2o01xnd6htdKvHajj0KoA7rbU3AbgZwN3GmDsA/A8Af2Ct3Q9gBsDHL4EODw8PD49LxGpK0FkALiohxX8WwJ0Afpbb7wfwXwH86YUSkKRmaflhsy2fQDv33ZKh0OVt6cjldzAudohPCFbKk6JZDffdJRJDK23tdK7AyLR+N3GDVME4fBis8PPbUFyZYeYjVm5SDS6TZhWHlBDF1UNiqw2mxJlUdXk8t1YtgULEpriCEaaFbaF+06rS+8EbqAjI1i1S1GBxge6VYu69UhVj2ug4ZQacPSvGxUnOLpjOCIfp5qvSIPfG7codLM/FI8YmxuRqZqe39m/GUgQp7qssc+rcFjUXnk8T15Z2LqmKpU9xzhebl0WrN4jzD9J6bblfztsSqyyhzqiog7CqdbpXqSwSWY2Dh3oKZE6rqvmLme5uVWAlYtfAouLyhza7vCNEW1kbfzMcUFYRuqvsIpkJls9a2ZqM0Ln6qRwx/FzV6+0GahdgVywKHVXmZtMpeW0V8kR3KtUe8JVIjcqn12XINMoV1AX8RCp6KGTXxEYjvbQL2JTTKij3yQ6+1kl/fCstFaRZ8jVdYgLVpQMvFqvSoRtjQi4QPQHgQQCvAZi1NsmccxrA9mW+e68x5pAx5pD2A/bw8PDwuLxY1QvdWtu01t4MYATAmwFcd56v6O/eZ6293Vp7ez6fP/8XPDw8PDwuChfkh26tnTXGPAzgrQD6jDERc+kjAM5cDAGS0L9D8vdOYZuMFlWHE7FaeuhkcGy7uzq3krFV4FLeBh0iOTsZOeMOka1J0yrzxyR0rPDzW1d5NpplEpysiuIDp5rVxSasEwFZ5K2qOpL1SpFpVHMUdNBVJVG6zmCqiaTv6mISu3buAgBcz/VXAWBuloyFzjdX58jIsUh6ekyrS0hcDYyoclw+mKYlKXBsXLZjuTjLpEm/GVbDYE+7ysWlkDUq9tHVC83oSu/G5VWheYzUxNTrJD7rKMUU5wfp65GiFyXn412jOXAGOgAwnOK3EascKs7PXhkS45DrqaZjPidrVnMG0LyooOqsQK3VhbYi74UsJ3qxKdkLixxxmVWGwQYbHgO7/KaMlBHaPV+RUrm4HDVRSx1Ous7Vql1YEN9tpxKMUjJ/LvOuVXl63HddqG1Pt/JzZzWIVpfU6y5aV2h3KqIc10U1RhtMeb2VisatvI5sdWqrIGiPjXDvPa1uaq5WL7sCzsuhG2OGjDF9fJwD8F4ALwF4GMCH+bJ7AHz1kqnx8PDw8LhorIZD3wrgfkMsWADgr621XzfGHAbweWPMfwfwNIA/uzgSOnDjHV0CW397tGHOZWGLO/Sxkhei5pBdRKfpJBVo16mEngtDp3wwF9rHSq6dNeUe12RjaKjCb112yNYeaICOq2gqLl84fpXDwkXJLjH7AkCaOeSugnCCO3eNAACGVEZDV1whFYn6zbm0ufXQ+T5KReKetg/vTtq6uvuYbtm+01Pk8tXbRfefPCtuesePUaGDwWFxfVwsujHcgqVY5IrzlaIqv2eIOw1CmaMsuyY6zqqq/AtrLBHp7Ikhu7stlCTzYaVB96obkk6agYwpz2XbrCoAkWH3QtsUA1rM7oSOEwxDbQQkA3IhK1yt+M3KvSLntsjur1lVxs5lF7TKNbbB3HV6hUjRmpL4HHeqizw4qaoTV5liSai/X0lhKcddK+Msc+Oh2pMu46YUtJF+XWBmU0nMLmNkPqefc+dAwcbRWMbS4PlQwkbyDtK0zXBkqLuVdg5I1lbnqrkMHPpqvFyeQ4ddb609BtKne3h4eHhcBfCRoh4eHh4bBGufnKtDvcxOShjt8Qks8YV2ftrqC3FydXvCLlGrtGfisktMq8tBjKhyjZh3larI1XtsSbNrkqPl0NpHayrPTqgpH1bjEvArkdq5N2tfcwtOxMQpWZt1ZRStl9uuT/pXYqWzaWfZ8Hn99dcn5/bto8RaBw7sT9oee+wJAMBzzz4vHbJhraePojwPXCNOVHVWYwykpVhClsX3nh5JgdqssG96EvEo6oFqlUTfYkmMekFqeV6mUeJowpqIww3+al2p32IW9+ts2ERN5ttFeRYGxCC3yKqR8qLU2sywoZE1Olioyrk61+0s5FRUrXX1UevqOjou8prVVARto0l9DKq5SvHWatZkz1R4DLZB18cq9XJ/ngzT09MSwZtig6BWqyzF4qIYNPN5Wr9OezhqSaPLahJWP6RVUQjniNBUsRRGp6VmuCcnlRghdZAL7Y+UiiTOZNoT1jrNUJMjijWJLn2uVq+4YemasN3dXdwHpzfWkbNslA2Cdr/8S4Hn0D08PDw2CMzl+FVYLbZt22bvvffeK3Y/Dw8Pj42AT33qU09aa28/33WeQ/fw8PDYIPAvdA8PD48NAv9C9/Dw8Ngg8C90Dw8Pjw2CK2oUNcZMAigCOHe+a69ybML6HsN6px9Y/2NY7/QD638M64n+Xdba89a+uKIvdAAwxhxajbX2asZ6H8N6px9Y/2NY7/QD638M653+TvAqFw8PD48NAv9C9/Dw8NggWIsX+n1rcM/LjfU+hvVOP7D+x7De6QfW/xjWO/1tuOI6dA8PDw+PHw28ysXDw8Njg+CKvtCNMXcbY14xxhw1xnziSt77YmCM2WGMedgYc9gY86Ix5te4fcAY86Ax5gh/9q81rSuBi3w/bYz5Ov+/xxjzOK/DF4wx6fP1sZYwxvQZY75kjHnZGPOSMeat63AN/gPvoReMMX9ljMlezetgjPmMMWbCGPOCaus454bwRzyO54wxt64d5YJlxvA/eR89Z4z5iqvGxud+k8fwijHmrrWh+tJwxV7oXPHojwG8H8BBAB8zxhy8Uve/SDQA/Ia19iCAOwD8CtP8CQAPWWsPAHiI/7+a8WugsoEO/wPAH1hr9wOYAfDxNaFq9fhDAH9vrb0OwE2gsaybNTDGbAfw7wHcbq29EUAI4KO4utfhswDuXtK23Jy/H8AB/rsXwJ9eIRrPh8+ifQwPArjRWvtGAK8C+E0A4Of6owBu4O/8iemUm/cqx5Xk0N8M4Ki19pi1tgbg8wA+dAXvf8Gw1o5aa5/i4wXQi2Q7iO77+bL7Afz02lB4fhhjRgD8JIBP8/8GwJ0AvsSXXO309wL4CXCJQ2ttzVo7i3W0BowIQM4YEwHIAxjFVbwO1tpHAEwvaV5uzj8E4M8t4TFQAfmtV4bS5dFpDNbab3FhewB4DFTgHqAxfN5aW7XWHgdwFOuwItuVfKFvB3BK/X+a29YFjDG7QaX4HgcwbK0d5VNjAIbXiKzV4H8B+E+Q+huDAGbVpr7a12EPgEkA/4fVRp82xhSwjtbAWnsGwO8BOAl6kc8BeBLrax2A5ed8vT7b/wbA3/Hxeh1DC7xRdBUwxnQB+DKAX7fWzutzltyErkpXIWPMTwGYsNY+uda0XAIiALcC+FNr7S2g1BEt6pWreQ0AgHXNHwL9OG0DUEC7KmBd4Wqf8/PBGPPbIJXq59aalsuJK/lCPwNgh/p/hNuuahhjUqCX+eestX/DzeNOpOTPieW+v8Z4G4APGmNOgFRcd4L00X0s+gNX/zqcBnDaWvs4//8l0At+vawBALwHwHFr7aS1tg7gb0Brs57WAVh+ztfVs22M+dcAfgrAz1nx215XY1gOV/KF/gSAA2zZT4MMEA9cwftfMFjf/GcAXrLW/r469QCAe/j4HgBfvdK0rQbW2t+01o5Ya3eD5vs71tqfA/AwgA/zZVct/QBgrR0DcMoYcy03vRvAYayTNWCcBHCHMSbPe8qNYd2sA2O5OX8AwC+wt8sdAOaUauaqgjHmbpAK8oPW2pI69QCAjxpjMsaYPSAD7w/XgsZLgrX2iv0B+ADIsvwagN++kve+SHrfDhIrnwPwDP99AKSHfgjAEQDfBjCw1rSuYizvBPB1Pt4L2qxHAXwRQGat6TsP7TcDOMTr8LcA+tfbGgD4FICXAbwA4C8AZK7mdQDwVyB9fx0kJX18uTkHVXb+Y36unwd581ytYzgK0pW75/l/q+t/m8fwCoD3rzX9F/PnI0U9PDw8Ngi8UdTDw8Njg8C/0D08PDw2CPwL3cPDw2ODwL/QPTw8PDYI/Avdw8PDY4PAv9A9PDw8Ngj8C93Dw8Njg8C/0D08PDw2CP4/cGTGc2y9d/4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "# 显示图片的函数\n",
    "\n",
    "\n",
    "def imshow(img):\n",
    "    img = img / 2 + 0.5     #  [-1,1] -> [0,1]\n",
    "    npimg = img.numpy()\n",
    "    plt.imshow(np.transpose(npimg, (1, 2, 0))) # (channel, width, height) -> (width, height, channel)\n",
    "\n",
    "\n",
    "# 随机选择一些图片\n",
    "dataiter = iter(trainloader)\n",
    "images, labels = dataiter.next()\n",
    "\n",
    "# 显示图片\n",
    "imshow(torchvision.utils.make_grid(images))\n",
    "# 打印label\n",
    "print(' '.join('%5s' % classes[labels[j]] for j in range(4)))"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "2. 定义卷积网络\n",
    "^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
    "代码和前面的介绍类似，只是输入通道从1变成3。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "\n",
    "\n",
    "class Net(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        self.conv1 = nn.Conv2d(3, 6, 5)\n",
    "        self.pool = nn.MaxPool2d(2, 2)\n",
    "        self.conv2 = nn.Conv2d(6, 16, 5)\n",
    "        self.fc1 = nn.Linear(16 * 5 * 5, 120)\n",
    "        self.fc2 = nn.Linear(120, 84)\n",
    "        self.fc3 = nn.Linear(84, 10)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.pool(F.relu(self.conv1(x)))\n",
    "        x = self.pool(F.relu(self.conv2(x)))\n",
    "        x = x.view(-1, 16 * 5 * 5)\n",
    "        x = F.relu(self.fc1(x))\n",
    "        x = F.relu(self.fc2(x))\n",
    "        x = self.fc3(x)\n",
    "        return x\n",
    "\n",
    "\n",
    "net = Net()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "3. 定义损失函数和optimizer\n",
    "^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
    "我们这里使用交叉熵损失函数，Optimizer使用带冲量的SGD。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.optim as optim\n",
    "\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "4. 训练网络\n",
    "^^^^^^^^^^^^^^^^^^^^\n",
    "\n",
    "我们遍历DataLoader进行训练。\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1,  2000] loss: 2.251\n",
      "[1,  4000] loss: 1.901\n",
      "[1,  6000] loss: 1.695\n",
      "[1,  8000] loss: 1.613\n",
      "[1, 10000] loss: 1.519\n",
      "[1, 12000] loss: 1.475\n",
      "[2,  2000] loss: 1.415\n",
      "[2,  4000] loss: 1.405\n",
      "[2,  6000] loss: 1.364\n",
      "[2,  8000] loss: 1.346\n",
      "[2, 10000] loss: 1.313\n",
      "[2, 12000] loss: 1.288\n",
      "Finished Training\n"
     ]
    }
   ],
   "source": [
    "for epoch in range(2):  # 这里只迭代2个epoch，实际应该进行更多次训练 \n",
    "\n",
    "    running_loss = 0.0\n",
    "    for i, data in enumerate(trainloader, 0):\n",
    "        # 得到输入\n",
    "        inputs, labels = data\n",
    "\n",
    "        # 梯度清零 \n",
    "        optimizer.zero_grad()\n",
    "\n",
    "        # forward + backward + optimize\n",
    "        outputs = net(inputs)\n",
    "        loss = criterion(outputs, labels)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "\n",
    "        # 定义统计信息\n",
    "        running_loss += loss.item()\n",
    "        if i % 2000 == 1999:\n",
    "            print('[%d, %5d] loss: %.3f' %\n",
    "                  (epoch + 1, i + 1, running_loss / 2000))\n",
    "            running_loss = 0.0\n",
    "\n",
    "print('Finished Training')"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "5. 在测试数据集上进行测试\n",
    "^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
    "\n",
    "我们进行了2轮迭代，可以使用测试数据集上的数据来进行测试。\n",
    "首先我们随机抽取几个样本来进行测试。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GroundTruth:    cat  ship  ship plane\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAB6CAYAAACvHqiXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJztfWmQJVl13ncz8+2vXu1dXdXd08t09+wwA8MAEkIIJHtAEihsAiMrpLGNYyIcIiw5FGEj64dMhH9IYYdkOULGMSEQSFYIYUACIywDA2KRNDA9K8z09DK9Vnd1Vdde9faXef3jnJvnvFp6qruarq7ifhEdlX0zX+a9N29mnnO+sxhrLTw8PDw8tj+Cre6Ah4eHh8fNgX+he3h4eOwQ+Be6h4eHxw6Bf6F7eHh47BD4F7qHh4fHDoF/oXt4eHjsEPgXuoeHh8cOwaZe6MaYR40xJ4wxp40xH7lZnfLw8PDwuH6YGw0sMsaEAE4C+BkA4wCeBvCL1tqXb173PDw8PDw2imgTv30EwGlr7RkAMMZ8GsD7AKz7Qi8Wi7avr28Tl/Tw8PD40cPExMS0tXb4tY7bzAt9D4CL6v/jAN58rR/09fXh8ccf38QlPTw8PH708NGPfvT8Ro77oZOixpjHjTHHjDHHarXaD/tyHh4eHj+y2MwL/RKAfer/e7mtC9baJ6y1D1trHy4Wi5u4nIeHh4fHtbCZF/rTAI4YYw4aY7IAPgjgizenWx4eHh4e14sbtqFbazvGmA8D+H8AQgCfsNa+dL3n2b/wBQCAsUnals1Qt0wg35tWqwkA6MRtOiabTffFCf3WJuKxY4IYABCEqs/tEu0D7ctkG+m+EO6aco446QAA2h3pW5IYvkDE/THpvibvkxYg4XEZI62tFo0hjqNVYw+4b61E2qrUDdRacdpWuvcxaHz4wx9Otzudzqpr3gxc9/nsir+6KdBt1Bq4Ru14Zdz8Jep4N89ykmt5a63Vb3f8xz72sVX79v8kz23cSdtmrl4BADQbsmYO3XkYANDXWwEAZELpTzZDCy+r23g9R0atsU4dAFAuZfgc0teIt0O1iOfmZgEAPT09aVsmk+Hz0nEmkHN0khYAIFhDdAuMNNaqZA6NIlqT+Xw+3ddq0Tk6/AwCQCFf4GtJ3/7g93636/x79+1Kt8tDR+l3oTy3lZ4yAGCpKeu6ujjD/aX7najFEPEgClEubcuH/ApTz236AHJTnMj5XVui2tw13Njp+jyXa6wdw/fPBPq9EK9xHP02l6P+ZgPpNyxtm6zMX23mOADgG0/9YNW5NorNkKKw1n4ZwJc3cw4PDw8Pj5uDTb3QbwZaLGVZW5dGlk5zKKVNAehLFkUseWuJg7+6JiONTSdVJPIFjFgCDLkpUucwCUnN6IgU4qTlRJ2jZUhyiUP6wrb0vjjgc8nX2rCUn1d9i1gyCiLqeNxuq450eEhyDieRhuH6FrIwDNfdd7NwoxK/no9UjlJSZOJEKstjsLLPaUwGIg3JWTYvoa+FcpHubWDl8WhWqS1pCbGfz9J5SwU6LlKXcWsnpxZZIcv3XY2lGbvjaF1l1TpxUxRFcm+d5B8oKd/NTY61Vr1MqrU2X1PgtFsLOW/AF8uwlOqkfgBoN5s8PjUWljpxjTWRWJHyO2E/nSsjz3QckoQeZJSEXl+mvsVV7oecr2npuLaSjBs8v0poR6tNWlTAz0S9Ju8W95zo8TmNOQjkObROs+HJ1BaBTifmY+Saxrj3k6yZ/n4ac67Qw+eXe5a4dZ2TfsTLZWwWPvTfw8PDY4fAv9A9PDw8dgi23ORi2SQBK6YOy2SUiUUlTNqkAoUFNmsotdVZGzQxkWWVqmNFpUnaYddxTnUCAGNXEHMADBM4NhTVsR6TbndlhtSzakvUqOVlagutnLcnz+SYIvUqRSKUCjkaZxK00n1Bal6RsbsRtJP1zQTahPDDqhO7kfN2mTfc8V26qdulTUQ05802zUek9eyYfhuata6drNG2MVxrLBGbvQJl9sqGdK1MIG25gM1pbp8iNJt1Ms2EoSLwIrrv7aYQqwHYxNahNmvkkYzZtJTNFOR4Nw9qjTlyOGazoY73mLl6FQAwMtQvx7N5JczKtUK+lptnZflBxMc3FUnsCNt2W9pWIrCyL+b+xuo5iA2NOd8j/RjcP0K/XZgDAJRry+m+VoPeEXFZnseklyLPe7Iy9+66AdtlW015vpwDRT4v9yWdUrUm3Dp2fwNl4+3wmBO9/Pjy2UjWbqHAxDGc2VBMOokz52qZ+iY4MXgJ3cPDw2OHYMsl9ChmyTyUr2PAkkYuVF9/xzjxlzLQzA//tKMlWEfyZEW62X3gLgDA4vw0AGB6RiSZTETSeAD5crc6ND11KwFRx8+TxGNzgwCAdigkT4slh+WF2bTt0iRLGnkleU3MAwDu2E3XHOzRUpxzZZSxO+Ejtqtdoxy0ZHwz3BVvipSf9ltpD+za2VHiTZs1pVNnzgAARnaLu1vC5PbwgEiYeSaSkk308VpzlGUpPOmIZBeydJVRhFyG24KY1lE2o6S+kF1jlfaVCejeJkZpZAm74zaYHFXrqcFjLxZlDYeOKdXiIc9DlV0qn3nm2XRXmzWF/sqb0rZcjp0D1BSkrrOsvQbKXdBY5xwga9ImjhhcX0LvQFwrA9BaT0JFCLOWFiptrcTsZqXI9/jZp9N9rWmS1kfvv0v6dpWeuaaReSvzwJbqRKzm1VhyrLEHg0JABkyK6ldKs0jnjdqsubRlspZKdF9yCwtpW7TvXgBAra83bUtY64r5nuUTIVZTi0AsbWG8efnaS+geHh4eOwT+he7h4eGxQ7DlJhenl5tI0uo6dbijIyiZgGqxGpxVZFMcO/VPmST4HNqv980//TMAgGf+/h8AAJfZ9AIA1Y6L/BRV7Pz4FADg7LikqMn1jwIA9o4cpGvmRK1ssbqYKUuWy06D1MSZqctpW7GfzDXjyxR92FDq80gPqYTFjKihcZvUZh0Mt5IOXIsUvRWRotc2zTD5llFRvexjXl8WEnx+gVTjyWkyVRV6RH0e5IhIHdXoSEAdPbpGZ1f0YuPIsnnPqnNk3OTH0u8Qjryntozy6247dTuRc4QVmgdjVdwB+zsnLho5lnW9vEimuXJRSMCA51tHbUYcWT3PZOjsopgSC+yn3VKWkVabrhVl9ZqhtpgjsTvK3OSitLPKx9rymk3i9c2AeuadCTFQY487PFZl6zBsEmkYuu+ZRNaCGSJTXG1J+tY+e5L6a8QslfB0VZ1/u3q+sm2OH7moSHmeD+1o0WDzadjguZJLormb+li/IqbVHkPPvOkdkvHxdduBI5pV7AXPd6hI9ijYvJnTS+geHh4eOwRbLqE3A/oSL9RUBBlLN/1lESsqTDJFLKFowip1O1IEjSNNa7W5tO3rX6K8MZPzJHFMLsv37PwlOu78ZUnxHuZJWo/DStpWqtCXOFOkfVFeJIMcS5H5QMYy3aIotdG9d6RtDSZrzpwhCX12XuWU2UPnPTAsmkKGXfeMchsT+YzHq77+Nrk+mTQNzFxDQNBSebCGhB6zFJawNKKjWV0E3tWZxbRtsUpjrev8HTUaTZAj8rlal3tbLrJEqvrm5P2NKiDXq6nkjHOxk/l2ZOiaLocJRyYql8OINcpIMY+hofmwsb57PD52BIiVa9vyEs3bBX3NyEVWizS5r0Lz5lwUX3jxxXTf6+67DwCQaJfKmOY3r116WVOo11gDjuT8HdYQw0icA9qcL6jZXD8ldqyk94TXsNUyJDsxtLR7I1+3d4nnangk3VfYtZ/6Y4WMBLte2qHdaVM9w7lZrlBeGCgX4Co/r3ZkMG3LJNSnhtLwS6wltpZofE2dY6fAEblVuS/RIGkPJqPcMjlfSw//NFQaQMfQ3JtAuehi89HeXkL38PDw2CHwL3QPDw+PHYItN7lcrZOaMdsWUvSbf/e3AIB7j4rp4qfuI7Khn/3VNRnjkvAESn2JmXxRXBrOnic/59k6qUK2OJDuC8tMvg2IeaDA9U9bKmVqi4m4Sj/1rVKWPk5dIRPK4pwiS1glzBfENHNhjsjYTIXUyakJqS5VvrIEANhdkeMLLlVvosi0FajWdHIzVjmVqulSC4cq0ZPbdulAVU4sBMnqb72LYtW2jmU2BzhytKCIswZH1E0ok8vUHG0nijBrsz2ltkQE8tS0zN/4pQkAwL1HDqVtdx7YS/1XfvkpOesifbWVxXVbhylcgyoN2eSXtMWcELCJr74gYwGbGywndQoLMvYs36usmm/TJlNbrM0UHA1tUiJWzE3VKpkWJifl+FKlzNdUicl4zlvLdFxe+cNfnSdi9dkfiBmmlKNrHj4kcxqx6adZo/VXiFQiqSatrVilkY7do9ZQ87ESaopdCtukK1aE96lnOcPmrtzpU3T6Z76d7uu8iU1VKg2t5RiR7JI8Gw3QPJQ53iPMyfFJic5vrCLqOTlez6C8gzKX2FyzTGsyMyLOD7hI+6KKmEUbV2l+w6K0JUfJN73Bib0CReJnOzQ5kbIl2mtw/BuFl9A9PDw8dgheU0I3xnwCwM8BmLLW3s9tAwD+AsABAOcAfMBaO7feOa7ZgV6SEmoz8m1pZ4l4nK2p5O8tciOqZNnNSxEpTiINQyFtGi2ScK8q/ml6ib7OxT4iRPqHhaisJiRpDEFF5TGB0sqI1NSokgTTWKbj9ytypcbS+FRLpGXD0tLCrJLKWFqp89c/zEq/JxdpGicWRCvYP8QayDW+4PN1GWi5SFpDoPJKuGIdXYK3I2tcEG5X2to1vvVruENemSCXzoEB0nYKeZF8mg0aczEnbbuHSdOySnyr1misJZZkWg2V7pQHvdyU8XXSPBvKjS51n3T7Vg2zS2K8lrdl3hUwUAc5CT2ntIIyk8+9TGYF7H4JADm+x3ktkLIWFTRkLaRFD7hQSmtR1lpPifb1D4gmeXactMAzF6+kbSdPPwkAmJsmiXS5IeeotanmTATlhsiS/wN3HU3b3vuzjwIA9vB6buZlnI1qlX8n16xwAXpTX8J6yISy/lz6a0eOApJCNlJyZXmOrtUZJzffitI2li7T9Vt5ica0oPeCuTKVtpXGmNCssOYJeZYK7C6bnZd+N5iI7kxPpG1ZnsPOIs1VblYcI9p11qYKouHMnyVnimxBJPSeUSJxXSooq1wUm44MV2u4lWxeRN+IhP5JAI+uaPsIgCettUcAPMn/9/Dw8PDYQrymhG6t/ZYx5sCK5vcBeAdvfwrA3wL4DzfSgbte9wgAYPypE2lbuZe+/o+89c1pWzEkO3OLJWQtfRrORhdbyffRs4vqVz//4ik5bx9Jh3v2kyuXVba4DEvhSXMmbWu1klXXCvmL+tILLwAAKipBfbFEkkFJ2dEuX5kE0J1nJmSpY4DdzebnxP43N0vbZyfENWtshFyyoqyKbliBqCKaQszSdVvX32PbZPoXYtd0wSpaIrVr+DA6AV55SKYBLi7fB5TraB+7frXb6lwstRXLYpN0ErrhYDGjXMRyBefepcqqMTHSZXNc1Te5Zqb7EN69voh+8dw57rfM99Iirbu4LZrCpUuknczxGqguiz151yBJ1eWSBAWFXJylpTIURpxrKOBcQlUlvTfcYFShjQuXiX85Oy48Q7VFv833sutcSSbGrcRSVmS3ifMUjHP58mTa9u1v/x0A4B7mKob7RCKtL5Pk78rDAUD7HsqnsrywvmKey8rYrZPWE6Uys4YTKDfbZQ4EXH749QCASvTGdF9tie5BW+V9MjmeG1WeMVOg61bZPVO727Y5X0pGPRt1nhvtNFhnu35tma5ZKshYGnx8rizP+UAPvXti9a5Y5rULdqMstFXGRu6T9jBu34T8STdqQx+x1jr95AqAkWsd7OHh4eHxw8emSVFLxst1Py3GmMeNMceMMcd0nmYPDw8Pj5uLG3VbnDTGjFprJ4wxowCm1jvQWvsEgCcAYGxsbNWLv9hLpoL9h4SgqbMF4o6Dh9O2IVbb58+eAwC0dXRZh0wXj7z9F9K2Ow49DAA4+MC5tO2Z58hM0l8mE8blKcnlErEbU04XV+DeLleF7JqfJbVzoJzRh1A/2KwyNCy5XFzRhuk5MaEYjqbsYZfHKFTECKvcr14cT9uG+0ktP7JXuU6twCf+5H/J+bkfGaX+lXtIZTx8UIjgN72O3Kpc2UurzEKOZLTavuJy7CiziiPssjk6vyY7s1kyoQz2K/dJVxtW1WhMc4Rk6ByNjpx/nknieZWqdGmBTABt7arJROYgu54dOSyEVcZFE+rC8EGXAaYL3/77p3i4qsCKI7LrshbOXSHiLq39qcSj/l4yWZQUSZzj4zLKlTFil7qAa4rWFKEZ8Tmsylt0ZZaI9LZit4s9zt2O8x0tK3dLvh+NhvS70kPnfcsbH0jbqpzyucEuuhcuiCnl1VdfpbErF7vzMzT39ZqcN8oJuQ8ApZI4GHR4HtqxvmdcaEaRgYZNUIURIj4XqzKWqws0dqPccVtcMzWrycV5+o3LBZXLynOwyGs8n1GvPpfWWEWKNjl6GVwzeKEua9Kl0SmqaNqevWTiDbUZMK2Hy/dK17Jwbw61KJOb4Ld4oxL6FwE8xtuPAfjCpnvi4eHh4bEpbMRt8c9BBOiQMWYcwG8D+B0AnzHGfAjAeQAfuNEOhDkiFi5PHk/bHnwjJeMv9coXP1wiAipmKSFS5bPOXCTi4m39B+XERQo+6SmpKu0RXavAboL5rCoVzl/nPWOjadPLLJlkFbmzyMTMwX2kURy9+9503+wsF7OoSIDCZXanMoqE6esnqXaBpU+d/6RQpN/Wl6Tfpy5wsIcitkYkdQUdX1PBT3XazqggnyUWcIuqLb7nbgBAwzJ5pCT0HEtKWqp1hSp0FsLeAdJGUuJJuTs6N6xQSeMu0kvLIglLK+c48OvSlCh8szOkEdXrItnFTZZEVc4Xl1Nk7z6ic+7YtzfdV0rXiiZ915fQnz9F/SgWRCOyrBE2O3JfejlrpiP/WkoKvrpM9yBUc9WTJ42sEwsJbpgEDNm3zUQSqJarkmTZagvZOjvryFBdLo3+tjhHzFJV5qrF7qz7hsX1cbCfFo8LXAKA2TnKAzPYR/14+PX3pfvG2TV1oS5r+JVxui+BWtcHVzBpkcp0WuihZ25ZlZSLWKWJVZbBiINvAl6TiXK3NFzwJlLXdFvtlsowyVp2xJK31ogcGRorLdCVtuuoVZkpMGkZr87a6nK/ZDpKU2CPAZ2xMR+7DJ18LbXkXGBdtxfx5rOjbsTL5RfX2fWuTV/dw8PDw+OmwUeKenh4eOwQbHkul0yeCJpGQ6vPXL9RRVAWS45kIlOArjdajkhl+uQTH0/bfv6ffZjOoaLbslxL0RXLOHhoT7pvapYIrsayqM27d5Hfui4Y0OQ6j4cOE2F752Ehcxeeo1qO1SVRKx2p01ERcnU2ifRx/cHYStRabz+pix1VkSAMaHzjl8UUMfI6dOED/+SfSh+ZLCyp/DGOhCkoU5VLLbG4yPlVOmIKyDBJFyn/W8uqa135Z9uEzueqomsiNuLjMxkdgbrabOP8bxuc/6SkcmT0cz6duCV9y4c0rvkZMRmMXzoHADjMRHoYKNOSdRXtVYrha7j8LrJZz2rikWMLCqHMx959d1L/XZrgK7LWptlUNDIi9VFzQ2QGqs6LP3fCkbC9/WSvyOUklqLBQ651xOSS5+cgbssaC5lcdEVfMllVaCNP24+8QUwoR/eP0flbstbPvkrjevXEywCAt75JCNN9++j4Cy9KzqF27HIqrV9TNKv6keWauokVM2eBSfCOSlO8xJGyMROf+V4xFY2U2ASmyEO3rrW5IoSrmUp/dWGOtWD52dQml5h93V2a4kBdM+sMPSpRVJPfKTp3VMQmxxicP0YXXeHnRtd11abXG4WX0D08PDx2CLZcQjccQVZTknGDJcyMzuMwwy5FnK8lg/l032gffTFPHZeo0Mvjp2mjJqXfzo+fAwA8tJuiU/fsF2ZxbIokpOppkUIGciQd9vRJWalXXz1L1xwj6X5+UaSnNn/pJ68qCcyRJco1scYSuuHcDpoKKbnsjYlEfmYNzUdr+grWQ9IWCSKVUNT+cpbOW8jLnNY5U16tTf04d+acXJNJ0TsO7k/bzl6kufzS3zyZtrU5w2We87UU1flddF1vRaIO+3pJynroIVExhodIKr1zL81poNwFnZTliCtAyK76LpHexkbpXo3tIVJbZ/CrsWtbl8ZyDVEmw0T98K6xtC3PhPT0tLiTVjlq2YX7NVQEaO8wra09yvW2p5fGWRkSqX2GifSYJba2qujmXCRrikhstR3hKRpL1mX0zNE9zljRoHbx3A/3yz3IM8E33C8sZoVd+2YuXAAAnH/1XLpv9wCt/4XJp9K2DJPhrXD9V0ikcpeEnEUyr/K7zE8RwTu7LDlUrk7Q/Pb30Pq//17RFDKsnTcVIdxmDUET+m79u6IvgSLqnZSsSyfGKRGrWcvu3EA6kyvSc8gzF/Hxeu2632Sc5qQfdD59oFww42u40m4UXkL38PDw2CHwL3QPDw+PHYItN7mkqW+V+jI6ROqWVt+//iL5hPdzkv0jA6IC5XNMCkXii3116hydvikRb3fcSX7qIZ+3WBECamiECKuZWVFvF5gM1YXNd+0idTlic1BDkZcu6VJdmQc6/OOOOkmjyak5O/Q9HVQquOFag1kjY8kxaRTb7kg8jb/6P19JtxNO2B8oH94yE8w9yvxx4AiNeXiQTAyDoxJFOsB9yqvkUvPHyRz1/eNSd7VuXTEN+n+k1OEK//bwHWK2eesjb6BrlcTHu8Rqu9N4W2pOO+xbXVsQE1ub/bgLRelbXx+ZGyY5Gdq0KpJR4IjFkd0yz8WiikFYgX42sYXKnNDkQh5GyUCzM9SnxUVOg6xMhCFHGJ6/JAmwKotkLuntlTgF53/eZKcAowjCnItmLMl9L1gXWapzAdMzUSqwOdKKOWbvIM1LURGU1UXqd0eZclzxj4NsIjr+ypl039GjlIgLigC9fJl80/P9YvYC9HY3CeiKrSTK/LHEMR1Xr4opcX6Oznvyxe8BAF554R/SfYcPU8zHgcP3pG39Q2w2UuYKlyraFTvRhoww9WFXfUsLvUibq5ErhXQU6crHa149jaxeg21PSdeu5Hd8VnW/9bvkRuEldA8PD48dgi2X0F0UV29ZCKu+Hto2KmfIoiVJY3qOvpRDPdL1EhM6cSCSybnL5wAAI/2SDH8/f+GdO9j3npHo1EsTJMn3lEVqz7Bb1UunL6geu0hH+ttUX9VljtDrUwUJOix2TkyqBPw91KeIXaOKRZHAXP4TtIVYjavUt5Fd6+dyefq5H6TbhQwRlM2mELZZJvXe/JY3pW3nL5GkPcOc1P33iWtblgnNWlOk/AxrNm94gxCaDY5EzLI0eeSQROvexylWx4ZEIq0U6d4myk314hWKUpya4+Ie01fTfVUmy+fnRUJvcQrbjHLBdLlkXCRxWxGUxT6at/sh4+vtXX8unaRdU5GooXEl/EQriDkVa8QRyIkV+Sibo/MPDUnkcZnXeF65gvZyvyO+Z9qd07JrYEe5k/ayS2egoisTThMbuejKpkjevZxAxnZEa4xZ62mpSMc6348ir83zV2T9vfwqaX/NpkSgths0vzbU1Pv6cFJtPi9jv/suilQ+fI+4D9eWSFp/6VlyAX7umBCx3/4WaYjHX5a1fvSeBwEAR+4Sqb2vn9abI4vDrj66+V0j97ImW13JvM7qso8uejRWJGqSuk+uj6701MaVzZQ1rFNs3yi8hO7h4eGxQ+Bf6B4eHh47BFtucnHRe7t3iU+4qzGYKHJxdC+p8sfYlDJvJEWtDUkt7x0S4rG3wj6geVGtD7DJpcwpe//4E3+a7qvxtRbrQqbV2A9YZ9rczZGcjVlS/6o5fU0yC71yQvzhJyfJfLCookf7+uiElRKpz6EisTIcvRfWLqVtwyXa35sXhU4lIQUAXL2o/OcHyGy0d6+QgPe+7gidPyfneOl5Ip5GWA0uq2pGU1xfsVQRk9VghY5776NvT9sCduju7aXjhgbFf36WUw2fPS/zsTBPZqDFBYmOXWLyeZ7TFM8uSgRohwnejEprnOUKQYGKrOut0Lj6OLK0X5mncmzSyhbEtLVcF9J5JQbZh1z79pe5+kyi0r9mApqPXeyvblSUbJZ9pp0pCADyHC0Zqjy7zsSSVmlSJhfng1+rytpxEYs5tSgtm19qCzTfl87JfM+y83NfQY4f4RTD+byuwcsmlIjMTVFRyPOrXN9z36g8cz1czWuxuT6Rl6i0uC6Jlw10G/UtVL7pfYOUhvZt76C1e/iwmPC+882/BQCcPSvPRvU5fm4XxST3wOuo2tG+fXQunZ467tAaj1XfEjbtdlXpSuvnur+yy9Xb1QS5s5Zon3dHkKbX6iJF+R2nzDbahHOj8BK6h4eHxw7BlkvojgSs9IuE3ompW7lI3MCOcmGGY8+Q5LWYkQi8xJC0N7JHvvQvHyd3px/7yX+Ztv0DFy6oVklKbLekwMXUFeeKJ9+4Za4BGKmovP6AJPg9BTrHwlWRhjohScYju4RYjdnVq64kwkadJNIqk2+dRCSwdoMi5XZlRBIcK5Mk1exI20oJ/dLJl9LtRSbOfv4f/Zu07dFHKTnm174u7o27mCzcVeQoUuUKl+fouZFekdR6eDuv3AU7LNU4SVTnrLlygiSpC1PiutfiQiVRXtLE9vQQibyLJcZ2azURlVFFClzOC537oqeHxlKp9PA+VaeS8+lMTsr9bjTWr55VZOm0rYjbArtg9lVE60nSVM5EaBZUndSU9FLSYWK5TctRrriI+6vIug7f704sfV2coTHoBzfDEvryAmmDE5clOnpkgMbSV5Jo5xpL14nSFDp8RkfE7uGCDQBwF9cZffBeKRpy8gw9L899XxwLVkKnjA64AEUQidadYaeAWEVXuvSzAZPER44KAZ+wm+/ExOfStrlpGuuppmh1k5eoPvGdR4h0vec+OceuESKpI/Vu6bS5+IZKqRtzjVx3H9csiNKVU2b1/jRFM8+DPkVaTEaJ/l3RqDcIL6F7eHh47BBspMDFPgB/AioEbQE8Ya39A2PMAIC/AHAAwDkAH7Av2Ih9AAAgAElEQVTWrl8CfB243CX9QyJBdPhr3gikMEK+zJIGZyi8cFGCEd72JnJHayzLF7PYQ26CE5ck98bpk1TtvOOqgStvpirbbXsGxc1sYYEko96ySKR3HaXcEk+/8AoA4NnjZ6UfP/UeAN1ZIs+cJgl+XmVsdC6PjTpJ5vtHRLIrcBDJwIBIxjYiyaHTWt+tqaFKgT3weurjO9/1zrRtsI9s2z/+ZmX/ZsmuhzWFSlmk5pCLNriq9IDYanXRgYU5sttWWOJJVAaZQ3fdDwDYtVcyUs7OkWbT0yeujC5zn7GrK7I7O6wrjQYAy2xTtqpkmCuccHGCbP9OCwKANhf/0PldiqX1A4uqrE31qAIXLshoSuXpWeRgp4SzMh52ATgA+jj/SZjR0idtay2mxfXMasydNJrS706L5sqoghi2SceXlMbS10caTiFLNu7IyDrpY+2ut0fWZIvPUVPZJFuc4TTgQJd+pZkVOUvpuOJpWLjGfXcdSduuKndTOpfmA9hervqW5d2JfhBZcnU25pbS1vbuOwAAOHDgQNr29CTd744qj3d1ap77Q9L78eMvpvtc4NSdd0q/R0bIbbKnR/gicIBfo8U2d/XsZVgj00FEzm1RxxVZo10jaVTp6dOCGILwJhS42IiE3gHwG9baewG8BcCvGmPuBfARAE9aa48AeJL/7+Hh4eGxRXjNF7q1dsJa+yxvLwE4DmAPgPcB+BQf9ikAv7D2GTw8PDw8bgWuixQ1xhwA8BCA7wIYsda6vJdXQCaZ60bCNRp7B6SoQbVOak4tFhXFEWCuVuTJl5QrXI1Um3JJcpFw7QGcPylq4iUmi976Vkqfq9OS9nA63IExcZO6MEtmlXpTJbcvkXpbGSbS6KEeqV15ldXxc+efl7HUyDwxvyDX2jVMqnGvpf7sL4ur364KF4UwYkJxKVNLSoUVpz/CobsfTLc/+Cv/msYXi1p+4jQRk4lROXCYPG2z+jc7r5LWJC6PjdCvrrB6AiG2lhapJ+EkqcaXVT1QV6gkaQjZVGIC9swpMYWd5ZStzu1vYEjmw5kHFhaE9JqZJmLQKhNKwO5wJnB5TVTkMROweZ06eHklrSzIsYvkzLSM5dU5uqaLsgSAvn4iv0dHaem3VFRhu0Vmm8RKHxfZLFZX5qCYIzhDNmfp2pXOrJIvyVgK7K7YUGs3YSKxVGY3WLVOshwlqQlkRzA3FAlo+DhHSrZVEZPxGbKk1lQNUkcq7h6V9b8SoTI5pNvqmjA8X13ufO43ZtU+F2Xa0yPmoJSs7Cpe4kx4dK2lObmPz3EK6pdeeDptGxik+7h7txDBu0cP8DXJDDOoTLHDXNDXKOLd3eeOMgN2mDRN3Ra16yObu6wyv9lkpYnm+rFhUtQYUwbwOQC/bq1d1PsszeCaBl5jzOPGmGPGmGO12vqeBR4eHh4em8OGJHRDKQA/B+DPrLWf5+ZJY8yotXbCGDMKYGqt31prnwDwBACMjY2teukvcSKRgspUl2aeS1S5NCZThgZIejsZSDa4qVmSfGZC+cL1lukrevf9QnScOUeSoCsioInKI0eIJDly8M607fwESSQvvfT9tG1mmoNUuAhCv3JVG3+JJPqJafneGSZ2QxXgNLqP3L/28xf7jh6RwPJcyqrZ0IEPJFFpt6qVeP8v/fN0u383SU0v/ECkYEcutZQUEDNJ50qtaVLGlfaKtQTBbUGXGMC5UzgL5vSMuCg6tzsVS4K+Sh/3RyTd2RnWRlhKnJ4WArTJ2klHuX3GXAYwVLlcinma55xzadQV2V3yHoj0VFBZJFdinoney5fE/a/EZPXdquCCy0hZ5Pw0jbpoVXNz5N7abss4a5xrpajcPnsrtO5LOfpbUGRnxFJnrEjRTqfF51XZO135s7QYgyqawFpuWz15UcikXqJcaTmb5MxV0kSmZ8TF02VFnFP5dJymlesRbWoljNUSOv3VRKFhqVbnOEklbf7rCEgAqC9TP65ckYIYly/T9kJRjsvwOnIkf0nljylGdJwmyC9xUY1T5+SdUq9TEZdOTOcaGpZiJw88QAGKRw6LRD88TGuh0ivOHbkCaRIWfH317HXSJI6KmL4VpKihnJIfB3DcWvt7atcXATzG248B+MKme+Ph4eHhccPYiIT+4wB+GcD3jTHOOPwfAfwOgM8YYz4E4DyAD/xwuujh4eHhsRG85gvdWvsdrJ8V8l2b7cCZ06Tm3HFE0l/mA04D2hLiKmK1SYgRIVHLXLTh7rvFD/hrX/kyAKC2IP7qxUEir06Pk3Vo314hUQ/eRYUXckqNP3QH7Z+fFff6l7luacKEy/ickEeLTOY2YjEfLc6TWWeXIlzOz1DbwD4yP8zklE90wiSqMq/YiGspJqK+r/Sifu75Y+n2i9+n766BmHJcvoxIF2FIU8Fm+BhR1SNOt6vTnbp8KlnV34D91ENL+ypZiZIN2CzVDpV5gCNnldswspxrpV1j/+iqmKxaTBqatooeZZtPS5HmMUeDVpfo+KK6j8O91I9ImTqcZWMtanRgmNZJvyo84go0RGo+lpaJmFxepv7mcmIucaSiTr86NkJkeC4v5gFHhlrOJ1JtSI8aTDjPz0l+oZlZ8vWuK/POPZymOMO+/d0FHbjeqVpPTa6FOp5GR4sPeYvNWbWqnH9hnkyPWRX16sb+5Ne/nra9/c0PoQuqeEPi/Ms7KkKTTTLKHR4mNQfRvlBFzr7w7DMAgOU58XcfZP/6ixPSVmEf+iw/N4mKsK6U2R9exQdkIy4MklNxGAGbcefIzHTurERiz8/RvD17TOXu4biNffskmnaMC8aMjtGzPzYi75sSp+k2BVXvNFg/NmKj8JGiHh4eHjsEW57L5fnTJC3fcf8jaVsC+joaTQLyF36RCZr5eSFtBgfIZe89j/5U2vbg6ymPw2c+/5dpm+G8DL1cfX3PmLhclZmsCzsimQzspukZPShS1gIXJ3j2eZKCJ5aVu1SGCNjeUSGKhg5TW1dhBHYTPMFFO05fEQk2y+xRXUVGVnkaOolIFe8W4REA8O1vfjXdrnHmuWxGlS4rOlJWbnloOX+Hq5Ke0RI69SOfU4Qtu/1lVZa+qERjzWdpnDmVj8KlCjEqS6Qjt9uqcEaDCc9UqtURdny8Lm2XhvgqibivRNu9JRpTuSBScC5D58sYuY9GuR+uRJtJOu3mGLFLZdxF9Lnyezx/SjTOsxRer8o465xhsq58Tp0mFGScG5us+RPHXwYAnD93Lm1zUc5WuUOOjZIDwABnvKwrbzK3PT8nhOYMk751pQG7nEPOE21+UbSkgOe+GMnacflirlwRDXilhN5WRTUcKW86cg4Xlaqd9SyozZGoy8syWa6Yyl1HRZt/w4MPAwCeeVGKXjz1NGURnefiKHFH7sGuUSI33/a2t6VtEd/nc+fFxfmppygX1P33UhR6pVecKyZ5zJOT4gDg1u7uEXFvPHjwAF2fHQuqS+L26RwMMpFoBY01chhdL7yE7uHh4bFD4F/oHh4eHjsEW25yOblAKv10rFKPZkgFD1pKRUlcDT76OzYqNoef+DEiNPMZUUMP7qfIz599/wfTts/+5V/Tta7QeScWRNlrNE4DALIQlXe2Ttunz4taCVaL7DCZdPpHxPyQ1hVU0ZgJmycSIyYAl4xqgSM58xmVhIxT2FaNSi7FZKRNtErWrZ6NDEv03ESdCKI4FjW7wnVOI9W3xWkie5cWq9wvUU0Tpy6vFb2mzCqZAt0Hm6Hru8RqABCwzaWokpW5yvRxe7U5DZwEymTFdpFncrOgzB8DPaSm7lMxAHtHyf/X8Z7NhqjqgaX1FKnIvr4Krbua5NpKcfIkpYS9775707YCm1D0dARMPyYcHTipomRdsrdmXZk12IQYK7PKocMHAADDu6j/uvBChs08fSpRliNUdZlM50P+yglKG7usCmK4fTqGIWGTUnVJ5qjG/axxNGtLmcRcMY0Lk0I8uhqv8TXqYNquCFDrNlK4KE8VxIrEEal8qwqq3u5PvONdvEt+4IpXHH1QTLb3v5Hq5rqyq4GiiV0BlkOHJN4k4jk9cETS7I7dQURzgSOOe5XJxY3LFXABxKyya1jSgLtkXyGbqgLF/sbs4NBWdrrErD+XG4WX0D08PDx2CLZcQj8xT9+UL3xHojEf3E/Syu6sEAZFlhJGd9MXcHRIpJY7DzG5aUWqmOC8Kp/49F+nbc88TySTi0TtCry0jpSSc8Q5ukasiT52BewwwdoJFGnoZlOVkmq0+LzqSxwxQRqyNGZVrpMOU0QZ9TV3pcha7fUjyWxbJPreEkkcS4pYbccktd19z/3ymzGSVqY4OnBKRQcuc14Xna7BSZY2lvOWIpJC7n49pSW9rErLXV0kDaDeEomxzoUldFRqjl0pS6yJ9KncJcNcwX10TCSfw3vIrXBXTsTUZXZ1nGW3vjAr81csEQleVhG5g5y/4/JZIcIc2izdN5ZFwwkcGalETFe8ImbXxFOnTqb7lhYcMS2PmCsCEinxOuGQwYAjbaFcMQdZq9Jka41TLtfrMqcXL453HaeCD2HZxbPWknvmpOvqtGjAGe6nK/nXUZGUVXZb7ChXSYm0XF+qrCvtJGQXzMiqCF5+XjsqgrfD8+DOr8vYOYG/ozQcVw6upXKojN3B+ZgSTlGbqCIS/JyfvSCuoPWWywOkCqb0Huy6/tyCXDNiibtUOSCDdfmQFmTMlydn+RzU8ZxKB+4CYE1Z1kdjbv2yiBuFl9A9PDw8dgj8C93Dw8Njh2DLTS7LrIZ87VlRV0++StGj736jkFJ3jpFqf/YMRWq+/U1iOsizqr7UEnXuM39D6TGffVkSLNVclBqbPAKVqtSpRYGKbnNmklipc002hbRZJTTKt7nJEZeaDIqi1fUvi5xIKAtXgTzdhZhJRZ0Uq8MEYrZHqvyszIU2c1kSccVtUt3qSh2uXaTEZAOqwvowp5XNcJWcgsqiVQ9dBRZtl1qtZtfqZKZ5O1eNuu8eSV514QKZM2bmJdK26cg2RaZFTHQXmMUaUgRoX6nEV5Z7cGWaxnJiWpI0GSa2KrvIjFSoCGFaZBJVp+UtK5JrJQp8z1rKrOHI6q46mc7/nM0VlYpEL+fZp79cElIv5HEVVbSpM3GceoUSuy3MiilggSM6Y+VznslyxKpaTznW3w3PX01Fm04xcVdrijof8hj6e2U9tdg8V2Mn+Y5K/pWk5hWd/5Xnw6wvE37rW9+QsXSoalApkvmIed21lVnFEfMuIZl+ltps2tLPoyMcG01pi9MKWJyKWtUPHegjc265rCtm0Rg0v2vS8bmEZyqik8ccKBNKxEm/ArP6ODeErvAKw++PohwfNNhcqAjv64WX0D08PDx2CLZcQh8covwWs3PyeZzgqLa/57qdABC39/MWfQmHd0uUpwnpC/y9YxIt9tdfp0ivZiISAfhLHQSrv2MxS45WfaadO5qWElyUZ4YlA6M/p5yHQpNerhalzj0T8vVDyxKHVZoCS/labB/dTdJkT0VJlbVuCX336EC6PX5hnMekiwnQ9tmTJ9KmBXYndFevKrfIKktDSdzFHNPxqphAq0kS3bPf+QoA4B0lGef9PM56r0jLjgTUUcANJuwWOHpTk7PnX6FovOm6RC42MnT9wi4Zc/9ukrhyFRpTqCJFi+z2lysKyW7C9Ze+c42NO3IPXJRx0lHaGo/dkaIFFUkZsNZYVzlRmrOkLV7QxSl4HlwKWZcvBxDyPJNXWgFfotWS+VuaI4m80Vjmv0JkuzuVV2u+XecUvKr+qyMw3V9NRjr3wo7STixLtdnM+kR9XkUqt0O+Lyoldo6dDhLl6urcNgO+piahE853o7UCFzGbWBUFzKO2rm6nUSQ0375A1cWNQk5Z3ZTI1pQg5eHpmqVt1pi11u3WjFHPxsr3TEtFvVo+R0O9PnIhaVNjY/txo/ASuoeHh8cOwZZL6E6azagsgJ0GSVdnJ0Uqa1Yp2OPtb6AK8oU+yZmwwMUgvvldyThYZ9tvW2W7y7HbmJM+1qqgFCppIf3YKttajiU740SlQB2fIymkoMqfORentgqkWWKpzQVlNJUk2NvPLpujkii/zP6QdRUIsvJTfMdRyeS2yC581fFpdQRn3VPuaLN83SyPuaXs5WK3Xe2W1lWQgHHqRcqfcXFJJJ/hgOajS8NhqWVZ2euvWJIKT7NNdVzlAKkVWcO5QwoMjBwkCSbfJ66r6X1gqalcFk2hyPb0QK0xew3b7yLnCaotidvi1GVak42G9M2Vj3N5PPQ9dppeoIKZMhz45ngVQDJcRmxz1y6KbbYj63wwzSatnSXlHuduW6nC7rBKMrRtmufmsqx1VyRjQUmkTjJ39mmj7OWJXR1c5nLbmGT9oiuJuo/LVeJRiqG+B/Q3VovZBUC12A2301GufFzIwyppXLJaynPYYRt67LRBda9dUJUWnq2lfjYbOrdN3HW81txtyufEqs0FFeoiMd3XDFu635w7p18XvqHtMXgJ3cPDw+NHHv6F7uHh4bFD8JomF2NMHsC3QDUVIgCftdb+tjHmIIBPAxgE8AyAX7ZWhWpuECnJpInBkFTHliJtJpdJLXr2BBFL76mJCrRkyRRxaU5MEnlWuTs1OUeDVUxXAzJSUXxuX5dbmnFuT3KcDbpTzmZy4oK2zK5eLZWC15lftNnBmViqHLFa7hPzSj/ngmiplJ+vsEtbRrlrvXGFVlbpF4JweITyq0wok0uq/qnfNNms4upNatfA+BoRgF17+MRtVtmr05LvI8hxSmLlMneZr/E8RB0/HfF8lEmNL+2TIhnDY5STZ5CLTgBAjl0BW6onls0CuYir3EeamHZtirS8hm/YlXPkQqursDsV3OiIX07f66q/a3U7y+YdncfG7deEY4dNDMvLXPO1qXOusMuc0S6EtC6yqhjDyJ4xPgdFdC7OiZtohwtWWEVCO3NKraXNMM6c4XzssOr4jBq7KzxRqykz4ApcvChOCqcmqB8lVSM0YltR3FWSg+bURYMmiqjPcq4f3eZMNLFObcTz7EhLo3KkOLJV27ZcPhh9X5x7bRK7KFJFdrKJsitnkyvgYVdHtrpftlWeqHiA1sWeB8Q1u9fd0k2kdNmIhN4E8E5r7esBPAjgUWPMWwD8LoDft9YeBjAH4EM33g0PDw8Pj81iIyXoLADnZ5XhfxbAOwG4UvOfAvCfAHzsunvgyAZdOICDXxKV98HlUzk7RRLBJz7z5XTfO99BSe7PXhbpsOqCBdQ3K+My1bGUUFRuR1kuXFFfEunaERdWkZYZJiidBKiJMCcJJopAqbOLmm5zx/WxVD2okuJfnaHAkvlpyfA4f56CqQ4fOoj1UMiLxJbjAJaMymcSMzmmP/6dVHLh8emd15ASuigyloaWeXyvKKmvl8vTvdKQQgAvsfYyUxHJdXAfjWv0IEnjfcoFM8dukIHKx9HmtRJGqpQbS8RRGmQjx6fStXYpuwYpGibsuqdcR1P3Qn1e1tYC6yQ2OUeTXTA7bVlPTuLWFecdHHmeyeoSgVw2UJPKvBbzOeX+V6DfzM7QNXUWxQxrnKGuLs/aaEdLkytIva5AGlfwQ2k9y1xEpVaVfDArEVhVvtBJq7FItU4b6ApOCtlt0TrXQKVpsWSs4qzSubfKNdHdCCs+iimcFK5dizt8/bZyCkj4HWRdiUD1PKR5mVRHDFaPxTL53eEAxorKR7T3AXLuiIzc7/mTnM9qr2ij14sN2dCNMSEXiJ4C8FUArwKYtxJGOA5gzzq/fdwYc8wYc2wtrxIPDw8Pj5uDDb3QrbWxtfZBAHsBPALg7o1ewFr7hLX2YWvtw0WV29jDw8PD4+biuvzQrbXzxphvAHgrgD5jTMRS+l4Al26kA4NcqbyhChJUOZItG4o/t0ur6XyJv/m9F9N9Z7m+4XxVmJHZZVKbFbeIEqvvHVa7cqp6vVPV8wWVJyJwPsKi2juf2Q6bGIz2T2UVLFYV6lvsJ1tQ+Ttckv2BITK1tBQh3OSCDvWcXDPh6EFdEX4l2iqis8r5OHr65JqNKqnZuoBCzOphmrFVpW41q60CKaxKD2yZUKqyj/C3VVGS8zVqm1H5KqIRqoA+unc4bTs4TNuDvTQvgYo2rbIq21DEVsSqv675meco0Iirr+cLIjzkeO51FOa1kKyRR8Qpo1aZfiyzyalJR53DRRrG2mTA60ivO7fGHEnbZfVK3HoSUjlm8rmVkXtb57S2ztSSaAKUc780lHbsxmW1L7Y73pkrVD8iHottCZE9N0NmtHZr/TXZUX7oMR/XCjQh7PL66KIo3MTPUqDugUuRm2jTCJvFEpVu2hHSzvqhj3cmM23lSZx/uDKxOTNTaprR/uVsFoImbJ3ZRr0P2pzGeuAuKqax58C+dF+D65G++orEzhTabNmWIPjrxmtK6MaYYWNMH28XAPwMgOMAvgHg/XzYYwC+cOPd8PDw8PDYLDYioY8C+JShhAgBgM9Ya79kjHkZwKeNMf8ZwHMAPn4jHWiw1JlTn5YmS0iZUKTUDn8oXcL+oCBS3DkmQwNF2nRYeuooQrPBGeWqHKmpiR8nNZWyIsUVmCgNlFThCMdCka6vc2pc5Ux5iXJPipgQ6a8Iabl7gLSS3buJ/JuviiSzyJkJlxckSrGPCx1MX9WRn0PQaKsq9mGWxt4/LNdsl2kuO22V2S5xf5kwVRK6G7KOGEylN83+OeKOsxG2VQ6VZi/1+84+IXn6Byi6s1yRpVcu0n3LMeHcUPlSWuzmaJV0HTp3U90P3s6wpqXdFl3xBk2w2Wuwvg129Yu0u6pzhdOujzx2V+hCr6eVkjd3gLqqIzl57p3bYKwiL9s8D6HSzNqcDyRW7rWlJmk2TjLXuXaadZbu1ygVl6wR8ev6Een55n7PTkr+oDZHrOpbsAp66JzzJcjKNTMu22ncVZGDf8pzpU5nXYZCpSHmWQPprwiR7krOuYIsek5DdjHNKQ3Y5Wnpio7l++IiZ5cWVR4WXp5JJHO0wKkUoyHpx/6jRHz2c/T3pVdOp/umT1NG2Uj1LX+NvDgbxUa8XF4E8NAa7WdA9nQPDw8Pj9sAPlLUw8PDY4dgy5NzOZUwp5IYFR0x0hZV07mZJuwFrRMGJayedVqKxIpdCk1NbNF2kqbolO/Z3CyZOmbVNStcGKFXRWFW2Hc9DzLHuOrdABCxShiqWpdNTubkCiTo4zo1rtVYU0mM5md47MLm5jkisXGN6MZQqWt9g2QOKpeUH3qTTVDK5NKJnW+68z1Wicb4Wx90pQNlM4JKLhWxCl1kE0dPj4pg5CIC5ZyQ2yX2Tc/mRF1t8eYy+83XFcHriNu8Um+zofPZFrU5WGHO0Pe9xaRXNqtIrMz6c+mifwNl1sg4U582l3Df3Ax1FW1PIwdV8qp4NTHtIqVdoYtWS+57nU0tcV1FdDIpWlJmqUIvqfQdHme7IecI1rCJpP74miB34SBsiiqpGI0q14ZdXBQzoLNY6TWzEmFHzTHX7UxUhLAF9TeEShnM2xJVqwhNY7v+AkDCyfdqkSTyk2hvl/5azTdHczfa0je31k2XL3vaST6TCkXl62vCu8KpnIePSqxIwO+qE09/l645JSbTkO+fLlSylgnseuEldA8PD48dAmNvwldhoxgbG7OPP/74Lbueh4eHx07ARz/60WestQ+/1nFeQvfw8PDYIfAvdA8PD48dAv9C9/Dw8Ngh8C90Dw8Pjx2CW0qKGmOuAqgCmH6tY29zDGF7j2G79x/Y/mPY7v0Htv8YtlP/91trh1/roFv6QgcAY8yxjbC1tzO2+xi2e/+B7T+G7d5/YPuPYbv3fy14k4uHh4fHDoF/oXt4eHjsEGzFC/2JLbjmzcZ2H8N27z+w/cew3fsPbP8xbPf+r8Itt6F7eHh4ePxw4E0uHh4eHjsEt/SFbox51Bhzwhhz2hjzkVt57RuBMWafMeYbxpiXjTEvGWN+jdsHjDFfNcac4r/9W93Xa4GLfD9njPkS//+gMea7fB/+whiTfa1zbCWMMX3GmM8aY14xxhw3xrx1G96Df8dr6AfGmD83xuRv5/tgjPmEMWbKGPMD1bbmnBvCf+dxvGiMecPW9Vywzhj+C6+jF40xf+mqsfG+3+QxnDDG/OOt6fXmcMte6Fzx6A8BvBvAvQB+0Rhz7626/g2iA+A3rLX3AngLgF/lPn8EwJPW2iMAnuT/3874NVDZQIffBfD71trDAOYAfGhLerVx/AGAv7HW3g3g9aCxbJt7YIzZA+DfAnjYWns/qJbPB3F734dPAnh0Rdt6c/5uAEf43+MAPnaL+vha+CRWj+GrAO631r4OwEkAvwkA/Fx/EMB9/Jv/Ybry6W4P3EoJ/REAp621Z6y1LQCfBvC+W3j964a1dsJa+yxvL4FeJHtA/f4UH/YpAL+wNT18bRhj9gL4WQB/xP83AN4J4LN8yO3e/14AbweXOLTWtqy189hG94ARASgYYyIARQATuI3vg7X2WwBmVzSvN+fvA/AnlvAUqID86K3p6fpYawzW2q9YSVL/FKQk8/sAfNpa27TWngVwGtuwItutfKHvAXBR/X+c27YFjDEHQKX4vgtgxFo7wbuuABhZ52e3A/4bgH8PwGX5HwQwrxb17X4fDgK4CuCP2Wz0R8aYErbRPbDWXgLwXwFcAL3IFwA8g+11H4D153y7Ptv/CsD/5e3tOoYueFJ0AzDGlAF8DsCvW2sX9T5LbkK3pauQMebnAExZa5/Z6r5sAhGANwD4mLX2IVDqiC7zyu18DwCAbc3vA32cxgCUsNoUsK1wu8/5a8EY81sgk+qfbXVfbiZu5Qv9EoB96v97ue22hjEmA3qZ/5m19vPcPOlUSv47td7vtxg/DuC9xphzIBPXO0H26D5W/YHb/z6MAxi31n6X//9Z0At+u9wDAPhpAGettVettW0Anwfdm+10H4D153xbPdvGmH8B4OcA/JIVv+1tNYb1cCtf6F5pGKAAAAF3SURBVE8DOMLMfhZEQHzxFl7/usH25o8DOG6t/T2164sAHuPtxwB84Vb3bSOw1v6mtXavtfYAaL6/bq39JQDfAPB+Puy27T8AWGuvALhojLmLm94F4GVsk3vAuADgLcaYIq8pN4Ztcx8Y6835FwH8Cnu7vAXAgjLN3FYwxjwKMkG+11pbU7u+COCDxpicMeYgiOD93lb0cVOw1t6yfwDeA2KWXwXwW7fy2jfY37eB1MoXATzP/94DskM/CeAUgK8BGNjqvm5gLO8A8CXePgRarKcB/G8Aua3u32v0/UEAx/g+/BWA/u12DwB8FMArAH4A4E8B5G7n+wDgz0H2/jZIS/rQenMOKqn8h/xcfx/kzXO7juE0yFbunuf/qY7/LR7DCQDv3ur+38g/Hynq4eHhsUPgSVEPDw+PHQL/Qvfw8PDYIfAvdA8PD48dAv9C9/Dw8Ngh8C90Dw8Pjx0C/0L38PDw2CHwL3QPDw+PHQL/Qvfw8PDYIfj/168Wu0MvjO8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "dataiter = iter(testloader)\n",
    "images, labels = dataiter.next()\n",
    "\n",
    "\n",
    "imshow(torchvision.utils.make_grid(images))\n",
    "print('GroundTruth: ', ' '.join('%5s' % classes[labels[j]] for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在我们用模型来预测这些样本：\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "outputs = net(images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "outputs是10个分类的logits。我们在训练的时候需要用softmax把它变成概率(CrossEntropyLoss帮我们做了)，但是预测的时候没有必要，因为我们只需要知道哪个分类的概率大就行。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted:    cat  ship  ship plane\n"
     ]
    }
   ],
   "source": [
    "_, predicted = torch.max(outputs, 1)\n",
    "\n",
    "print('Predicted: ', ' '.join('%5s' % classes[predicted[j]]\n",
    "                              for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "结果还不错。接下来我们看看在整个测试集合上的效果：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of the network on the 10000 test images: 54 %\n"
     ]
    }
   ],
   "source": [
    "correct = 0\n",
    "total = 0\n",
    "with torch.no_grad():\n",
    "    for data in testloader:\n",
    "        images, labels = data\n",
    "        outputs = net(images)\n",
    "        _, predicted = torch.max(outputs.data, 1)\n",
    "        total += labels.size(0)\n",
    "        correct += (predicted == labels).sum().item()\n",
    "\n",
    "print('Accuracy of the network on the 10000 test images: %d %%' % (\n",
    "    100 * correct / total))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "看起来比随机的瞎猜要好，因为随机猜的准确率大概是10%的准确率。因此模型确实学到了一些。我们也可以看每个分类的准确率：\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of plane : 67 %\n",
      "Accuracy of   car : 58 %\n",
      "Accuracy of  bird : 46 %\n",
      "Accuracy of   cat : 44 %\n",
      "Accuracy of  deer : 42 %\n",
      "Accuracy of   dog : 48 %\n",
      "Accuracy of  frog : 58 %\n",
      "Accuracy of horse : 48 %\n",
      "Accuracy of  ship : 76 %\n",
      "Accuracy of truck : 53 %\n"
     ]
    }
   ],
   "source": [
    "class_correct = list(0. for i in range(10))\n",
    "class_total = list(0. for i in range(10))\n",
    "with torch.no_grad():\n",
    "    for data in testloader:\n",
    "        images, labels = data\n",
    "        outputs = net(images)\n",
    "        _, predicted = torch.max(outputs, 1)\n",
    "        c = (predicted == labels).squeeze()\n",
    "        for i in range(4):\n",
    "            label = labels[i]\n",
    "            class_correct[label] += c[i].item()\n",
    "            class_total[label] += 1\n",
    "\n",
    "\n",
    "for i in range(10):\n",
    "    print('Accuracy of %5s : %2d %%' % (\n",
    "        classes[i], 100 * class_correct[i] / class_total[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "在GPU上训练\n",
    "----------------\n",
    "为了在GPU上训练，我们需要把Tensor移到GPU上。首先我们看看是否有gpu，如果没有，那么我们还是fallback到cpu。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cuda:0\n"
     ]
    }
   ],
   "source": [
    "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
    " \n",
    "\n",
    "print(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "用GPU进行训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1,  2000] loss: 2.189\n",
      "[1,  4000] loss: 1.842\n",
      "[1,  6000] loss: 1.684\n",
      "[1,  8000] loss: 1.597\n",
      "[1, 10000] loss: 1.515\n",
      "[1, 12000] loss: 1.475\n",
      "[2,  2000] loss: 1.407\n",
      "[2,  4000] loss: 1.385\n",
      "[2,  6000] loss: 1.358\n",
      "[2,  8000] loss: 1.334\n",
      "[2, 10000] loss: 1.332\n",
      "[2, 12000] loss: 1.282\n",
      "[3,  2000] loss: 1.237\n",
      "[3,  4000] loss: 1.227\n",
      "[3,  6000] loss: 1.229\n",
      "[3,  8000] loss: 1.215\n",
      "[3, 10000] loss: 1.211\n",
      "[3, 12000] loss: 1.177\n",
      "[4,  2000] loss: 1.138\n",
      "[4,  4000] loss: 1.100\n",
      "[4,  6000] loss: 1.144\n",
      "[4,  8000] loss: 1.136\n",
      "[4, 10000] loss: 1.112\n",
      "[4, 12000] loss: 1.104\n",
      "[5,  2000] loss: 1.045\n",
      "[5,  4000] loss: 1.066\n",
      "[5,  6000] loss: 1.060\n",
      "[5,  8000] loss: 1.060\n",
      "[5, 10000] loss: 1.042\n",
      "[5, 12000] loss: 1.064\n",
      "[6,  2000] loss: 0.977\n",
      "[6,  4000] loss: 0.990\n",
      "[6,  6000] loss: 0.990\n",
      "[6,  8000] loss: 1.003\n",
      "[6, 10000] loss: 1.031\n",
      "[6, 12000] loss: 1.032\n",
      "[7,  2000] loss: 0.908\n",
      "[7,  4000] loss: 0.928\n",
      "[7,  6000] loss: 0.956\n",
      "[7,  8000] loss: 0.964\n",
      "[7, 10000] loss: 0.973\n",
      "[7, 12000] loss: 0.996\n",
      "[8,  2000] loss: 0.867\n",
      "[8,  4000] loss: 0.872\n",
      "[8,  6000] loss: 0.926\n",
      "[8,  8000] loss: 0.933\n",
      "[8, 10000] loss: 0.940\n",
      "[8, 12000] loss: 0.953\n",
      "[9,  2000] loss: 0.860\n",
      "[9,  4000] loss: 0.868\n",
      "[9,  6000] loss: 0.864\n",
      "[9,  8000] loss: 0.875\n",
      "[9, 10000] loss: 0.903\n",
      "[9, 12000] loss: 0.924\n",
      "[10,  2000] loss: 0.788\n",
      "[10,  4000] loss: 0.827\n",
      "[10,  6000] loss: 0.865\n",
      "[10,  8000] loss: 0.853\n",
      "[10, 10000] loss: 0.903\n",
      "[10, 12000] loss: 0.895\n",
      "[11,  2000] loss: 0.788\n",
      "[11,  4000] loss: 0.802\n",
      "[11,  6000] loss: 0.804\n",
      "[11,  8000] loss: 0.848\n",
      "[11, 10000] loss: 0.843\n",
      "[11, 12000] loss: 0.861\n",
      "[12,  2000] loss: 0.740\n",
      "[12,  4000] loss: 0.774\n",
      "[12,  6000] loss: 0.806\n",
      "[12,  8000] loss: 0.816\n",
      "[12, 10000] loss: 0.826\n",
      "[12, 12000] loss: 0.848\n",
      "[13,  2000] loss: 0.717\n",
      "[13,  4000] loss: 0.764\n",
      "[13,  6000] loss: 0.774\n",
      "[13,  8000] loss: 0.792\n",
      "[13, 10000] loss: 0.799\n",
      "[13, 12000] loss: 0.826\n",
      "[14,  2000] loss: 0.701\n",
      "[14,  4000] loss: 0.742\n",
      "[14,  6000] loss: 0.751\n",
      "[14,  8000] loss: 0.797\n",
      "[14, 10000] loss: 0.793\n",
      "[14, 12000] loss: 0.799\n",
      "[15,  2000] loss: 0.685\n",
      "[15,  4000] loss: 0.705\n",
      "[15,  6000] loss: 0.746\n",
      "[15,  8000] loss: 0.775\n",
      "[15, 10000] loss: 0.753\n",
      "[15, 12000] loss: 0.782\n",
      "[16,  2000] loss: 0.673\n",
      "[16,  4000] loss: 0.671\n",
      "[16,  6000] loss: 0.731\n",
      "[16,  8000] loss: 0.765\n",
      "[16, 10000] loss: 0.778\n",
      "[16, 12000] loss: 0.764\n",
      "[17,  2000] loss: 0.659\n",
      "[17,  4000] loss: 0.704\n",
      "[17,  6000] loss: 0.702\n",
      "[17,  8000] loss: 0.712\n",
      "[17, 10000] loss: 0.750\n",
      "[17, 12000] loss: 0.758\n",
      "[18,  2000] loss: 0.640\n",
      "[18,  4000] loss: 0.683\n",
      "[18,  6000] loss: 0.716\n",
      "[18,  8000] loss: 0.732\n",
      "[18, 10000] loss: 0.726\n",
      "[18, 12000] loss: 0.764\n",
      "[19,  2000] loss: 0.593\n",
      "[19,  4000] loss: 0.675\n",
      "[19,  6000] loss: 0.700\n",
      "[19,  8000] loss: 0.742\n",
      "[19, 10000] loss: 0.725\n",
      "[19, 12000] loss: 0.721\n",
      "[20,  2000] loss: 0.630\n",
      "[20,  4000] loss: 0.646\n",
      "[20,  6000] loss: 0.678\n",
      "[20,  8000] loss: 0.696\n",
      "[20, 10000] loss: 0.725\n",
      "[20, 12000] loss: 0.756\n",
      "Finished Training\n"
     ]
    }
   ],
   "source": [
    "class Net2(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Net2, self).__init__()\n",
    "        self.conv1 = nn.Conv2d(3, 6, 5).to(device)\n",
    "        self.pool = nn.MaxPool2d(2, 2).to(device)\n",
    "        self.conv2 = nn.Conv2d(6, 16, 5).to(device)\n",
    "        self.fc1 = nn.Linear(16 * 5 * 5, 120).to(device)\n",
    "        self.fc2 = nn.Linear(120, 84).to(device)\n",
    "        self.fc3 = nn.Linear(84, 10).to(device)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.pool(F.relu(self.conv1(x)))\n",
    "        x = self.pool(F.relu(self.conv2(x)))\n",
    "        x = x.view(-1, 16 * 5 * 5)\n",
    "        x = F.relu(self.fc1(x))\n",
    "        x = F.relu(self.fc2(x))\n",
    "        x = self.fc3(x)\n",
    "        return x\n",
    "\n",
    "\n",
    "net = Net2()\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)\n",
    "\n",
    "for epoch in range(20):\n",
    "\n",
    "    running_loss = 0.0\n",
    "    for i, data in enumerate(trainloader, 0):\n",
    "        # 得到输入\n",
    "        inputs, labels = data \n",
    "        inputs, labels = inputs.to(device), labels.to(device) \n",
    "        # 梯度清零 \n",
    "        optimizer.zero_grad()\n",
    "\n",
    "        # forward + backward + optimize\n",
    "        outputs = net(inputs)\n",
    "        loss = criterion(outputs, labels)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "\n",
    "        # 定义统计信息\n",
    "        running_loss += loss.item()\n",
    "        if i % 2000 == 1999:\n",
    "            print('[%d, %5d] loss: %.3f' %\n",
    "                  (epoch + 1, i + 1, running_loss / 2000))\n",
    "            running_loss = 0.0\n",
    "\n",
    "print('Finished Training')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of the network on the 10000 test images: 60 %\n"
     ]
    }
   ],
   "source": [
    "correct = 0\n",
    "total = 0\n",
    "with torch.no_grad():\n",
    "    for data in testloader:\n",
    "        images, labels = data\n",
    "        images = images.to(device)\n",
    "        labels = labels.to(device)\n",
    "        outputs = net(images)\n",
    "        _, predicted = torch.max(outputs.data, 1)\n",
    "        total += labels.size(0)\n",
    "        correct += (predicted == labels).sum().item()\n",
    "\n",
    "print('Accuracy of the network on the 10000 test images: %d %%' % (\n",
    "    100 * correct / total))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "py3.6-env",
   "language": "python",
   "name": "py3.6-env"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
